[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:06 dist-test-slave-fljv krb5kdc[13234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:06 dist-test-slave-fljv krb5kdc[13234](info): set up 2 sockets
Jun 19 18:47:06 dist-test-slave-fljv krb5kdc[13234](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 18:47:07.417935 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:07 dist-test-slave-fljv krb5kdc[13234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:07.541397 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46545
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:46545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:07.652493 13247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:07.652777 13247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:07.652817 13247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:07.656879 13247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:07.656945 13247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:07.656963 13247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:07.656978 13247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:07.656991 13247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:07.662313 13247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:46545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:07.663370 13247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:07.664391 13247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:07.672741 13253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:07.672832 13247 server_base.cc:1034] running on GCE node
W20260619 18:47:07.672716 13252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:07.672741 13255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:07.673593 13247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:07.674696 13247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:07.675861 13247 hybrid_clock.cc:648] HybridClock initialized: now 1781894827675842 us; error 31 us; skew 500 ppm
Jun 19 18:47:07 dist-test-slave-fljv krb5kdc[13234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894827, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:07.678817 13247 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:07.679688 13247 webserver.cc:460] Webserver started at http://127.12.234.254:43715/ using document root <none> and password file <none>
I20260619 18:47:07.680254 13247 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:07.680318 13247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:07.680509 13247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:07.682550 13247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "873326cd59e84abfbe7e4450cdefa9f9"
format_stamp: "Formatted at 2026-06-19 18:47:07 on dist-test-slave-fljv"
server_key: "1203900de24c86b99c8bb942aa23aca9"
server_key_iv: "9ed9c69fc145a2a7122be4d9ea542e90"
server_key_version: "encryptionkey@0"
I20260619 18:47:07.683079 13247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "873326cd59e84abfbe7e4450cdefa9f9"
format_stamp: "Formatted at 2026-06-19 18:47:07 on dist-test-slave-fljv"
server_key: "1203900de24c86b99c8bb942aa23aca9"
server_key_iv: "9ed9c69fc145a2a7122be4d9ea542e90"
server_key_version: "encryptionkey@0"
I20260619 18:47:07.686834 13247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 18:47:07.689374 13262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:07.690716 13247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:07.690826 13247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "873326cd59e84abfbe7e4450cdefa9f9"
format_stamp: "Formatted at 2026-06-19 18:47:07 on dist-test-slave-fljv"
server_key: "1203900de24c86b99c8bb942aa23aca9"
server_key_iv: "9ed9c69fc145a2a7122be4d9ea542e90"
server_key_version: "encryptionkey@0"
I20260619 18:47:07.690940 13247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:07.738843 13247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:07.742101 13247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:07.742465 13247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:07.749362 13247 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:46545
I20260619 18:47:07.749382 13314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:46545 every 8 connection(s)
I20260619 18:47:07.750391 13247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:07.753281 13315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:07.758808 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 13247
I20260619 18:47:07.758957 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:07.758994 13315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9: Bootstrap starting.
I20260619 18:47:07.760274 13227 external_mini_cluster.cc:1428] Setting key 3829ba27c866ac93b6a1936880098683
I20260619 18:47:07.761523 13315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:07.762283 13315 log.cc:824] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:07.764462 13315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9: No bootstrap required, opened a new log
I20260619 18:47:07.768544 13315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873326cd59e84abfbe7e4450cdefa9f9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46545 } }
I20260619 18:47:07.768842 13315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:07.768973 13315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 873326cd59e84abfbe7e4450cdefa9f9, State: Initialized, Role: FOLLOWER
I20260619 18:47:07.769515 13315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873326cd59e84abfbe7e4450cdefa9f9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46545 } }
I20260619 18:47:07.769642 13315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:07.769706 13315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:07.769802 13315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:07.770780 13315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873326cd59e84abfbe7e4450cdefa9f9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46545 } }
I20260619 18:47:07.771112 13315 leader_election.cc:302] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 873326cd59e84abfbe7e4450cdefa9f9; no voters:
I20260619 18:47:07.771435 13315 leader_election.cc:288] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:07.771588 13320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:07.771803 13320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [term 1 LEADER]: Becoming Leader. State: Replica: 873326cd59e84abfbe7e4450cdefa9f9, State: Running, Role: LEADER
I20260619 18:47:07.772115 13320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873326cd59e84abfbe7e4450cdefa9f9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46545 } }
I20260619 18:47:07.772739 13315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:07.774905 13321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "873326cd59e84abfbe7e4450cdefa9f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873326cd59e84abfbe7e4450cdefa9f9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46545 } } }
I20260619 18:47:07.775586 13321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:07.775259 13322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 873326cd59e84abfbe7e4450cdefa9f9. Latest consensus state: current_term: 1 leader_uuid: "873326cd59e84abfbe7e4450cdefa9f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873326cd59e84abfbe7e4450cdefa9f9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46545 } } }
I20260619 18:47:07.775810 13322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9 [sys.catalog]: This master's current role is: LEADER
W20260619 18:47:07.778254 13334 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:47:07.778328 13334 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:47:07.778406 13336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:07.779071 13336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 18:47:07 dist-test-slave-fljv krb5kdc[13234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:07.784862 13336 catalog_manager.cc:1347] Generated new cluster ID: d304ee4444d047e0a4defc17b62996be
I20260619 18:47:07.784933 13336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:07.786580 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:07.763282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.12.234.254:46545)
0619 18:47:07.763750 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:07.763768 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:07.764968 (+ 1200us) server_negotiation.cc:407] Connection header received
0619 18:47:07.768043 (+ 3075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:07.768065 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:07.768473 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:07.768865 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:07.771329 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:07.772928 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:07.774398 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:07.775145 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:07.775300 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:07.780184 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:07.780191 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:07.780210 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:07.780244 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:07.782302 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:07.783056 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:07.783062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:07.783068 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:07.783210 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:07.783563 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:07.783567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:07.783569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:07.784141 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:07.784358 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:07.785383 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0619 18:47:07.785677 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":139,"threads_started":1}
I20260619 18:47:07.803773 13336 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:07.805006 13336 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:07.813576 13336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 873326cd59e84abfbe7e4450cdefa9f9: Generated new TSK 0
I20260619 18:47:07.814337 13336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:07.851346 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46545
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:07.971057 13341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:07.971349 13341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:07.971395 13341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:07.975752 13341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:07.975828 13341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:07.975898 13341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:07.981428 13341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46545
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:07.982561 13341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:07.983585 13341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:07.993681 13347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:07.993708 13349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:07.993739 13346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:07.994589 13341 server_base.cc:1034] running on GCE node
I20260619 18:47:07.994998 13341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:07.995668 13341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:07.996852 13341 hybrid_clock.cc:648] HybridClock initialized: now 1781894827996825 us; error 45 us; skew 500 ppm
Jun 19 18:47:07 dist-test-slave-fljv krb5kdc[13234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894827, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:07.999801 13341 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:08.000710 13341 webserver.cc:460] Webserver started at http://127.12.234.193:46443/ using document root <none> and password file <none>
I20260619 18:47:08.001276 13341 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:08.001332 13341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:08.001493 13341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:08.003656 13341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "4740b05c35364778be5b14bb92558d11"
format_stamp: "Formatted at 2026-06-19 18:47:07 on dist-test-slave-fljv"
server_key: "1c22a968aa02ba530f34fee70b5f0098"
server_key_iv: "8260649471aba8d3f9d3445d4fc02a39"
server_key_version: "encryptionkey@0"
I20260619 18:47:08.004228 13341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "4740b05c35364778be5b14bb92558d11"
format_stamp: "Formatted at 2026-06-19 18:47:07 on dist-test-slave-fljv"
server_key: "1c22a968aa02ba530f34fee70b5f0098"
server_key_iv: "8260649471aba8d3f9d3445d4fc02a39"
server_key_version: "encryptionkey@0"
I20260619 18:47:08.008496 13341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:08.011327 13356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:08.012696 13341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 18:47:08.012812 13341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "4740b05c35364778be5b14bb92558d11"
format_stamp: "Formatted at 2026-06-19 18:47:07 on dist-test-slave-fljv"
server_key: "1c22a968aa02ba530f34fee70b5f0098"
server_key_iv: "8260649471aba8d3f9d3445d4fc02a39"
server_key_version: "encryptionkey@0"
I20260619 18:47:08.012939 13341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:08.026724 13341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:08.030385 13341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:08.030586 13341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:08.031365 13341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:08.032390 13341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:08.032452 13341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:08.032506 13341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:08.032534 13341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:08.041263 13341 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:33601
I20260619 18:47:08.041323 13469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:33601 every 8 connection(s)
I20260619 18:47:08.042277 13341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:08.050016 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 13341
I20260619 18:47:08.050137 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:08.050508 13227 external_mini_cluster.cc:1428] Setting key 3608834280289079251ed4cd21752ab2
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894827, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:08.057389 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:08.044253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:52397 (local address 127.12.234.254:46545)
0619 18:47:08.044372 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:08.044376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:08.045484 (+ 1108us) server_negotiation.cc:407] Connection header received
0619 18:47:08.046455 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:08.046458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:08.046512 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:08.046584 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:08.047808 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.048506 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.049698 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.049949 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.050045 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:08.053097 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:08.053100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:08.053102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:08.053129 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:08.054977 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.055577 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.055581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.055582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.055685 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.056126 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.056130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.056132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.056384 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:08.056492 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:08.057050 (+ 558us) server_negotiation.cc:299] Negotiation successful
0619 18:47:08.057206 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:47:08.058250 13472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:08.044679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46545 (local address 127.12.234.193:52397)
0619 18:47:08.045338 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:08.045383 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:08.046224 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:08.046758 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:08.046766 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:08.047147 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:08.047635 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:08.047650 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.048642 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.048645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:08.049567 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:08.049576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.050118 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.050123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:08.050373 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:08.051001 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:08.051018 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:08.052898 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:08.055108 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:08.055162 (+ 54us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:08.055178 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:08.055477 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:08.055799 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:08.055802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:08.055804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:08.056014 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:08.056501 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:08.056509 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:08.056792 (+ 283us) client_negotiation.cc:769] Sending connection context
0619 18:47:08.056963 (+ 171us) client_negotiation.cc:240] Negotiation successful
0619 18:47:08.057275 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":112,"threads_started":1}
I20260619 18:47:08.059898 13470 heartbeater.cc:347] Connected to a master server at 127.12.234.254:46545
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:08.060163 13470 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:08.060726 13470 heartbeater.cc:511] Master 127.12.234.254:46545 requested a full tablet report, sending...
I20260619 18:47:08.062389 13276 ts_manager.cc:194] Registered new tserver with Master: 4740b05c35364778be5b14bb92558d11 (127.12.234.193:33601)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:47:08.063738 13276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:52397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:08.079392 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46545
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:08.191556 13475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:08.191812 13475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:08.191855 13475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:08.195888 13475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:08.195955 13475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:08.196017 13475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:08.201416 13475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46545
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:08.202530 13475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:08.203598 13475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:08.213225 13480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:08.213230 13483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:08.213320 13475 server_base.cc:1034] running on GCE node
W20260619 18:47:08.213239 13481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:08.213799 13475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:08.214525 13475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:08.215701 13475 hybrid_clock.cc:648] HybridClock initialized: now 1781894828215679 us; error 34 us; skew 500 ppm
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894828, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:08.218518 13475 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:08.219450 13475 webserver.cc:460] Webserver started at http://127.12.234.194:41085/ using document root <none> and password file <none>
I20260619 18:47:08.219993 13475 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:08.220060 13475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:08.220316 13475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:08.222434 13475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "ae94953836384b788077c47ab6d9fa40"
format_stamp: "Formatted at 2026-06-19 18:47:08 on dist-test-slave-fljv"
server_key: "b5e6655bda0e4b078b421a537b647ebb"
server_key_iv: "49614a097aab5bc7975327431f1f8986"
server_key_version: "encryptionkey@0"
I20260619 18:47:08.222990 13475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "ae94953836384b788077c47ab6d9fa40"
format_stamp: "Formatted at 2026-06-19 18:47:08 on dist-test-slave-fljv"
server_key: "b5e6655bda0e4b078b421a537b647ebb"
server_key_iv: "49614a097aab5bc7975327431f1f8986"
server_key_version: "encryptionkey@0"
I20260619 18:47:08.226845 13475 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:47:08.229436 13490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:08.230801 13475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:08.230938 13475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "ae94953836384b788077c47ab6d9fa40"
format_stamp: "Formatted at 2026-06-19 18:47:08 on dist-test-slave-fljv"
server_key: "b5e6655bda0e4b078b421a537b647ebb"
server_key_iv: "49614a097aab5bc7975327431f1f8986"
server_key_version: "encryptionkey@0"
I20260619 18:47:08.231047 13475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:08.241566 13475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:08.244853 13475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:08.245040 13475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:08.245645 13475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:08.246511 13475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:08.246559 13475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:08.246613 13475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:08.246639 13475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:08.254904 13475 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:33449
I20260619 18:47:08.254926 13603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:33449 every 8 connection(s)
I20260619 18:47:08.255859 13475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:08.265743 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 13475
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894828, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:08.265857 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:08.266106 13227 external_mini_cluster.cc:1428] Setting key 9fcc4f71f024612da1683079514e5491
I20260619 18:47:08.270807 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:08.257769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:60641 (local address 127.12.234.254:46545)
0619 18:47:08.257908 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:08.257912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:08.259082 (+ 1170us) server_negotiation.cc:407] Connection header received
0619 18:47:08.260013 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:08.260017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:08.260077 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:08.260224 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:08.261666 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.262254 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.263536 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.263787 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.263880 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:08.266602 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:08.266606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:08.266609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:08.266643 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:08.268421 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.268984 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.268988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.268990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.269084 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.269498 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.269502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.269503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.269707 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:08.269842 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:08.270490 (+ 648us) server_negotiation.cc:299] Negotiation successful
0619 18:47:08.270652 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 18:47:08.271674 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:08.258215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46545 (local address 127.12.234.194:60641)
0619 18:47:08.258911 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:08.258955 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:08.259785 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:08.260357 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:08.260365 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:08.260788 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:08.261468 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:08.261493 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.262385 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.262388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:08.263418 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:08.263428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.263944 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.263948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:08.264119 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:08.264709 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:08.264726 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:08.266387 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:08.268560 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:08.268566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:08.268582 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:08.268883 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:08.269180 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:08.269182 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:08.269184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:08.269403 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:08.269901 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:08.269908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:08.270230 (+ 322us) client_negotiation.cc:769] Sending connection context
0619 18:47:08.270412 (+ 182us) client_negotiation.cc:240] Negotiation successful
0619 18:47:08.270744 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":114,"threads_started":1}
I20260619 18:47:08.272871 13604 heartbeater.cc:347] Connected to a master server at 127.12.234.254:46545
I20260619 18:47:08.273113 13604 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:08.273646 13604 heartbeater.cc:511] Master 127.12.234.254:46545 requested a full tablet report, sending...
I20260619 18:47:08.274784 13276 ts_manager.cc:194] Registered new tserver with Master: ae94953836384b788077c47ab6d9fa40 (127.12.234.194:33449)
I20260619 18:47:08.275588 13276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:60641
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:08.297791 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46545
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:08.412727 13609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:08.413034 13609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:08.413080 13609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:08.417717 13609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:08.417814 13609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:08.417896 13609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:08.423879 13609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46545
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:08.425030 13609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:08.426147 13609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:08.436655 13614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:08.436616 13617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:08.436645 13615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:08.436738 13609 server_base.cc:1034] running on GCE node
I20260619 18:47:08.437356 13609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:08.438088 13609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:08.439289 13609 hybrid_clock.cc:648] HybridClock initialized: now 1781894828439269 us; error 39 us; skew 500 ppm
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894828, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:08.442464 13609 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:08.443579 13609 webserver.cc:460] Webserver started at http://127.12.234.195:38619/ using document root <none> and password file <none>
I20260619 18:47:08.444207 13609 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:08.444291 13609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:08.444509 13609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:08.446852 13609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "1310e0a5e2b04c099c1d9895d52c322d"
format_stamp: "Formatted at 2026-06-19 18:47:08 on dist-test-slave-fljv"
server_key: "aec028068ab711f040e7415d3e7d7e1c"
server_key_iv: "e730c7de776aa03bc51a5796f74bac3d"
server_key_version: "encryptionkey@0"
I20260619 18:47:08.447460 13609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "1310e0a5e2b04c099c1d9895d52c322d"
format_stamp: "Formatted at 2026-06-19 18:47:08 on dist-test-slave-fljv"
server_key: "aec028068ab711f040e7415d3e7d7e1c"
server_key_iv: "e730c7de776aa03bc51a5796f74bac3d"
server_key_version: "encryptionkey@0"
I20260619 18:47:08.451551 13609 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 18:47:08.454440 13624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:08.455974 13609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 18:47:08.456113 13609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "1310e0a5e2b04c099c1d9895d52c322d"
format_stamp: "Formatted at 2026-06-19 18:47:08 on dist-test-slave-fljv"
server_key: "aec028068ab711f040e7415d3e7d7e1c"
server_key_iv: "e730c7de776aa03bc51a5796f74bac3d"
server_key_version: "encryptionkey@0"
I20260619 18:47:08.456255 13609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:08.474133 13609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:08.477478 13609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:08.477682 13609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:08.478327 13609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:08.479202 13609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:08.479251 13609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:08.479306 13609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:08.479334 13609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:08.488034 13609 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:33125
I20260619 18:47:08.488077 13737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:33125 every 8 connection(s)
I20260619 18:47:08.488982 13609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:08.494776 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 13609
I20260619 18:47:08.494908 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:08.495157 13227 external_mini_cluster.cc:1428] Setting key 84ea022ca09d3bda6acd6b7714575436
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894828, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:08.503332 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:08.490985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:49551 (local address 127.12.234.254:46545)
0619 18:47:08.491137 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:08.491142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:08.492233 (+ 1091us) server_negotiation.cc:407] Connection header received
0619 18:47:08.493478 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:08.493481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:08.493530 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:08.493607 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:08.495046 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.495704 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.497363 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.497638 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.497728 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:08.500253 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:08.500256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:08.500258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:08.500286 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:08.501474 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.501935 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.501937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.501939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.502012 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.502346 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.502350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.502351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.502547 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:08.502622 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:08.503101 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 18:47:08.503206 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 18:47:08.504228 13740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:08.491483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46545 (local address 127.12.234.195:49551)
0619 18:47:08.492014 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:08.492063 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:08.493275 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:08.493774 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:08.493781 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:08.494285 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:08.494803 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:08.494818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.495862 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.495866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:08.497214 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:08.497226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.497795 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.497799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:08.497990 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:08.498599 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:08.498619 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:08.500079 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:08.501575 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:08.501580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:08.501594 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:08.501857 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:08.502075 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:08.502077 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:08.502078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:08.502260 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:08.502628 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:08.502633 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:08.502891 (+ 258us) client_negotiation.cc:769] Sending connection context
0619 18:47:08.503047 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 18:47:08.503371 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":141,"threads_started":1}
I20260619 18:47:08.505504 13738 heartbeater.cc:347] Connected to a master server at 127.12.234.254:46545
I20260619 18:47:08.505761 13738 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:08.506315 13738 heartbeater.cc:511] Master 127.12.234.254:46545 requested a full tablet report, sending...
I20260619 18:47:08.507380 13276 ts_manager.cc:194] Registered new tserver with Master: 1310e0a5e2b04c099c1d9895d52c322d (127.12.234.195:33125)
I20260619 18:47:08.508044 13276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:49551
I20260619 18:47:08.509711 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:08.535704 13745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:08.525707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56150 (local address 127.12.234.193:33601)
0619 18:47:08.525985 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:08.526002 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:08.526031 (+ 29us) server_negotiation.cc:407] Connection header received
0619 18:47:08.526270 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:08.526277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:08.526484 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:08.526662 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:08.527372 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.528148 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.528972 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.529365 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.529460 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:08.531955 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:08.531965 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:08.531979 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:08.532008 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:08.533714 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.534298 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.534305 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.534307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.534401 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.534690 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.534692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.534694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.534952 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:08.535156 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:08.535388 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 18:47:08.535543 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":61,"threads_started":1}
W20260619 18:47:08.536736 13384 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56150
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:08.562916 13745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:08.553921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56154 (local address 127.12.234.193:33601)
0619 18:47:08.554154 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:08.554157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:08.554266 (+ 109us) server_negotiation.cc:407] Connection header received
0619 18:47:08.554407 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:08.554410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:08.554456 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:08.554530 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:08.555348 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.555922 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.556867 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:08.557104 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:08.557199 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:08.559649 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:08.559653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:08.559655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:08.559682 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:08.561184 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.561617 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.561621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.561622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.561708 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:08.561951 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:08.561953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:08.561954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:08.562126 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:08.562282 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:08.562663 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 18:47:08.562794 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 18:47:08.563934 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 13341
I20260619 18:47:08.568667 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 13475
I20260619 18:47:08.573693 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 13609
I20260619 18:47:08.578639 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 13247
2026-06-19T18:47:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2143 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13752](info): set up 2 sockets
Jun 19 18:47:08 dist-test-slave-fljv krb5kdc[13752](info): commencing operation
krb5kdc: starting...
W20260619 18:47:09.537704 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:09 dist-test-slave-fljv krb5kdc[13752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:09.658214 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:45847
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:45847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:09.770819 13765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:09.771123 13765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:09.771165 13765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:09.775393 13765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:09.775465 13765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:09.775483 13765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:09.775501 13765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:09.775516 13765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:09.781085 13765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:45847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:45847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:09.782325 13765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:09.783475 13765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:09.792078 13770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:09.792178 13765 server_base.cc:1034] running on GCE node
W20260619 18:47:09.792111 13771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:09.792066 13773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:09.792954 13765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:09.794106 13765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:09.795284 13765 hybrid_clock.cc:648] HybridClock initialized: now 1781894829795256 us; error 35 us; skew 500 ppm
Jun 19 18:47:09 dist-test-slave-fljv krb5kdc[13752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894829, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:09.798028 13765 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:09.798962 13765 webserver.cc:460] Webserver started at http://127.12.234.254:32867/ using document root <none> and password file <none>
I20260619 18:47:09.799517 13765 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:09.799576 13765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:09.799753 13765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:09.801856 13765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "543b50d5a317401b8de06e7163a4d17a"
format_stamp: "Formatted at 2026-06-19 18:47:09 on dist-test-slave-fljv"
server_key: "100f40d0d41b6b2b98af7a2b4ab399d0"
server_key_iv: "7938c6885ea8bd67f21420ba47238e58"
server_key_version: "encryptionkey@0"
I20260619 18:47:09.802407 13765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "543b50d5a317401b8de06e7163a4d17a"
format_stamp: "Formatted at 2026-06-19 18:47:09 on dist-test-slave-fljv"
server_key: "100f40d0d41b6b2b98af7a2b4ab399d0"
server_key_iv: "7938c6885ea8bd67f21420ba47238e58"
server_key_version: "encryptionkey@0"
I20260619 18:47:09.806044 13765 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260619 18:47:09.808583 13780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:09.810001 13765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:09.810130 13765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "543b50d5a317401b8de06e7163a4d17a"
format_stamp: "Formatted at 2026-06-19 18:47:09 on dist-test-slave-fljv"
server_key: "100f40d0d41b6b2b98af7a2b4ab399d0"
server_key_iv: "7938c6885ea8bd67f21420ba47238e58"
server_key_version: "encryptionkey@0"
I20260619 18:47:09.810289 13765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:09.824291 13765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:09.827749 13765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:09.827919 13765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:09.834730 13765 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:45847
I20260619 18:47:09.834746 13832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:45847 every 8 connection(s)
I20260619 18:47:09.835776 13765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:09.838850 13833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:09.844624 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 13765
I20260619 18:47:09.844749 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:09.845007 13227 external_mini_cluster.cc:1428] Setting key 3a256afafe314101b28550016099b3fa
I20260619 18:47:09.845170 13833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a: Bootstrap starting.
I20260619 18:47:09.847863 13833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:09.848706 13833 log.cc:824] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:09.850775 13833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a: No bootstrap required, opened a new log
Jun 19 18:47:09 dist-test-slave-fljv krb5kdc[13752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:09.854686 13833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543b50d5a317401b8de06e7163a4d17a" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45847 } }
I20260619 18:47:09.854971 13833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:09.855027 13833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543b50d5a317401b8de06e7163a4d17a, State: Initialized, Role: FOLLOWER
I20260619 18:47:09.855507 13833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543b50d5a317401b8de06e7163a4d17a" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45847 } }
I20260619 18:47:09.855674 13833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:09.855754 13833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:09.855859 13833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:09.856748 13833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543b50d5a317401b8de06e7163a4d17a" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45847 } }
I20260619 18:47:09.857118 13833 leader_election.cc:302] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 543b50d5a317401b8de06e7163a4d17a; no voters:
I20260619 18:47:09.857473 13833 leader_election.cc:288] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:09.857590 13838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:09.857812 13838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [term 1 LEADER]: Becoming Leader. State: Replica: 543b50d5a317401b8de06e7163a4d17a, State: Running, Role: LEADER
I20260619 18:47:09.858130 13838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543b50d5a317401b8de06e7163a4d17a" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45847 } }
I20260619 18:47:09.858879 13833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:09.859833 13839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "543b50d5a317401b8de06e7163a4d17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543b50d5a317401b8de06e7163a4d17a" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45847 } } }
I20260619 18:47:09.859930 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:09.846503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57180 (local address 127.12.234.254:45847)
0619 18:47:09.846982 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:09.846999 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:09.847036 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:47:09.847731 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:09.847753 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:09.848147 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:09.848461 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:09.849317 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:09.850355 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:09.851234 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:09.851712 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:09.851858 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:09.854606 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:09.854612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:09.854627 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:09.854658 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:09.856491 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:09.856974 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:09.856981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:09.856988 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:09.857141 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:09.857433 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:09.857436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:09.857438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:09.858015 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:09.858288 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:09.858583 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 18:47:09.858892 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":161,"threads_started":1}
I20260619 18:47:09.859956 13839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:09.860105 13840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 543b50d5a317401b8de06e7163a4d17a. Latest consensus state: current_term: 1 leader_uuid: "543b50d5a317401b8de06e7163a4d17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543b50d5a317401b8de06e7163a4d17a" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45847 } } }
I20260619 18:47:09.860164 13840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a [sys.catalog]: This master's current role is: LEADER
W20260619 18:47:09.863831 13853 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:47:09.863932 13853 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:47:09.864019 13847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:09.864858 13847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:09.870369 13847 catalog_manager.cc:1347] Generated new cluster ID: 645e36a7471e4ecf99d992434bb85b17
I20260619 18:47:09.870443 13847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:09.896286 13847 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:09.897262 13847 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:09.908695 13847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 543b50d5a317401b8de06e7163a4d17a: Generated new TSK 0
I20260619 18:47:09.909601 13847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:09.949592 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45847
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:10.086678 13859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:10.086989 13859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:10.087035 13859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:10.091200 13859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:10.091282 13859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:10.091346 13859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:10.096875 13859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:10.097970 13859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:10.099021 13859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:10.108904 13865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:10.108989 13859 server_base.cc:1034] running on GCE node
W20260619 18:47:10.108948 13864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:10.108906 13867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:10.109632 13859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:10.110330 13859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:10.111488 13859 hybrid_clock.cc:648] HybridClock initialized: now 1781894830111461 us; error 41 us; skew 500 ppm
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:10.118934 13859 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:10.119958 13859 webserver.cc:460] Webserver started at http://127.12.234.193:41097/ using document root <none> and password file <none>
I20260619 18:47:10.120563 13859 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:10.120636 13859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:10.120867 13859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:10.123169 13859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "b64d1a3c85394a9e9bde0a327c217794"
format_stamp: "Formatted at 2026-06-19 18:47:10 on dist-test-slave-fljv"
server_key: "09963b2be4723c8b53f36b03c974a3cd"
server_key_iv: "0fd7f75fe0acd9d2f5c8895557beaa7a"
server_key_version: "encryptionkey@0"
I20260619 18:47:10.123832 13859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "b64d1a3c85394a9e9bde0a327c217794"
format_stamp: "Formatted at 2026-06-19 18:47:10 on dist-test-slave-fljv"
server_key: "09963b2be4723c8b53f36b03c974a3cd"
server_key_iv: "0fd7f75fe0acd9d2f5c8895557beaa7a"
server_key_version: "encryptionkey@0"
I20260619 18:47:10.128065 13859 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:10.130810 13874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:10.132189 13859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:10.132320 13859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "b64d1a3c85394a9e9bde0a327c217794"
format_stamp: "Formatted at 2026-06-19 18:47:10 on dist-test-slave-fljv"
server_key: "09963b2be4723c8b53f36b03c974a3cd"
server_key_iv: "0fd7f75fe0acd9d2f5c8895557beaa7a"
server_key_version: "encryptionkey@0"
I20260619 18:47:10.132434 13859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:10.150144 13859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:10.153548 13859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:10.153736 13859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:10.154402 13859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:10.155256 13859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:10.155303 13859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:10.155354 13859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:10.155380 13859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:10.164201 13859 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:37185
I20260619 18:47:10.164237 13987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:37185 every 8 connection(s)
I20260619 18:47:10.165104 13859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:10.166826 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 13859
I20260619 18:47:10.166935 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:10.167204 13227 external_mini_cluster.cc:1428] Setting key 23bc1101ce5816a179d94129e35e89e7
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:47:10.180068 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.166982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:37911 (local address 127.12.234.254:45847)
0619 18:47:10.167100 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.167104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.168288 (+ 1184us) server_negotiation.cc:407] Connection header received
0619 18:47:10.169797 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.169801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.169856 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.169934 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:10.171263 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.171790 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.173479 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.173723 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.173804 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.176578 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:10.176582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:10.176585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:10.176614 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:10.177968 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.178519 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.178523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.178525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.178605 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.178937 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.178940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.178942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.179143 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:10.179228 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.179754 (+ 526us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.179898 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 18:47:10.180899 13990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.167568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:45847 (local address 127.12.234.193:37911)
0619 18:47:10.168107 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:10.168177 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:10.169573 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:10.170129 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:10.170146 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:10.170554 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:10.171100 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.171120 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.171910 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.171913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.173359 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.173371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.173883 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.173887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.174093 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.174744 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:10.174768 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:10.176414 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:10.178080 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:10.178086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:10.178104 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:10.178426 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:10.178688 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:10.178691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:10.178693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:10.178840 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:10.179237 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:10.179244 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:10.179510 (+ 266us) client_negotiation.cc:769] Sending connection context
0619 18:47:10.179682 (+ 172us) client_negotiation.cc:240] Negotiation successful
0619 18:47:10.180030 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":152,"threads_started":1}
I20260619 18:47:10.182144 13988 heartbeater.cc:347] Connected to a master server at 127.12.234.254:45847
I20260619 18:47:10.182441 13988 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:10.183073 13988 heartbeater.cc:511] Master 127.12.234.254:45847 requested a full tablet report, sending...
I20260619 18:47:10.184971 13797 ts_manager.cc:194] Registered new tserver with Master: b64d1a3c85394a9e9bde0a327c217794 (127.12.234.193:37185)
I20260619 18:47:10.186645 13797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:37911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:10.195447 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45847
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:10.305272 13993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:10.305547 13993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:10.305601 13993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:10.309772 13993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:10.309854 13993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:10.309916 13993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:10.315408 13993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:10.316485 13993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:10.317492 13993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:10.327821 13998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:10.327821 13999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:10.328078 13993 server_base.cc:1034] running on GCE node
W20260619 18:47:10.327960 14001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:10.328613 13993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:10.329308 13993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:10.330476 13993 hybrid_clock.cc:648] HybridClock initialized: now 1781894830330448 us; error 40 us; skew 500 ppm
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:10.333287 13993 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:10.334383 13993 webserver.cc:460] Webserver started at http://127.12.234.194:43731/ using document root <none> and password file <none>
I20260619 18:47:10.334998 13993 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:10.335075 13993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:10.335279 13993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:10.337538 13993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35"
format_stamp: "Formatted at 2026-06-19 18:47:10 on dist-test-slave-fljv"
server_key: "9de83a9801dc3aa4084e2361966d7dfa"
server_key_iv: "f5279666b098ff4e7ddf10d720e0fb99"
server_key_version: "encryptionkey@0"
I20260619 18:47:10.338135 13993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35"
format_stamp: "Formatted at 2026-06-19 18:47:10 on dist-test-slave-fljv"
server_key: "9de83a9801dc3aa4084e2361966d7dfa"
server_key_iv: "f5279666b098ff4e7ddf10d720e0fb99"
server_key_version: "encryptionkey@0"
I20260619 18:47:10.342370 13993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:47:10.345243 14008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:10.346974 13993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:10.347132 13993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35"
format_stamp: "Formatted at 2026-06-19 18:47:10 on dist-test-slave-fljv"
server_key: "9de83a9801dc3aa4084e2361966d7dfa"
server_key_iv: "f5279666b098ff4e7ddf10d720e0fb99"
server_key_version: "encryptionkey@0"
I20260619 18:47:10.347263 13993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:10.368352 13993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:10.371654 13993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:10.371842 13993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:10.372444 13993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:10.373303 13993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:10.373349 13993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:10.373407 13993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:10.373431 13993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:10.381876 13993 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:46773
I20260619 18:47:10.381902 14121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:46773 every 8 connection(s)
I20260619 18:47:10.382828 13993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:10.392063 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 13993
I20260619 18:47:10.392186 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:10.392464 13227 external_mini_cluster.cc:1428] Setting key b7c210b22bf6108e2264094bbc4757d0
I20260619 18:47:10.396118 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.384571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:44855 (local address 127.12.234.254:45847)
0619 18:47:10.384752 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.384756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.385596 (+ 840us) server_negotiation.cc:407] Connection header received
0619 18:47:10.386893 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.386896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.386943 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.387016 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:10.388153 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.388686 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.389779 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.389996 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.390071 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.392563 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:10.392567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:10.392570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:10.392595 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:10.394011 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.394518 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.394522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.394524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.394612 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.394974 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.394977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.394979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.395178 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:10.395275 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.395810 (+ 535us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.395939 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 18:47:10.397044 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.384993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:45847 (local address 127.12.234.194:44855)
0619 18:47:10.385439 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:10.385483 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:10.386663 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:10.387172 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:10.387179 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:10.387527 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:10.388025 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.388037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.388801 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.388803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.389671 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.389679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.390109 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.390111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.390288 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.390835 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:10.390851 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:10.392390 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:10.394112 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:10.394117 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:10.394130 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:10.394427 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:10.394720 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:10.394723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:10.394724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:10.394886 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:10.395287 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:10.395293 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:10.395536 (+ 243us) client_negotiation.cc:769] Sending connection context
0619 18:47:10.395717 (+ 181us) client_negotiation.cc:240] Negotiation successful
0619 18:47:10.395994 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":124,"threads_started":1}
I20260619 18:47:10.398479 14122 heartbeater.cc:347] Connected to a master server at 127.12.234.254:45847
I20260619 18:47:10.398728 14122 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:10.399276 14122 heartbeater.cc:511] Master 127.12.234.254:45847 requested a full tablet report, sending...
I20260619 18:47:10.400418 13797 ts_manager.cc:194] Registered new tserver with Master: cea86eb6cdfa4b0f87c8ff38229f4f35 (127.12.234.194:46773)
I20260619 18:47:10.400991 13797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:44855
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:10.421281 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45847
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:10.533212 14127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:10.533506 14127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:10.533571 14127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:10.537890 14127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:10.537966 14127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:10.538051 14127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:10.543581 14127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:10.544725 14127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:10.545775 14127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:10.555765 14135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:10.555920 14127 server_base.cc:1034] running on GCE node
W20260619 18:47:10.555805 14132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:10.555845 14133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:10.556450 14127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:10.557147 14127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:10.558307 14127 hybrid_clock.cc:648] HybridClock initialized: now 1781894830558283 us; error 36 us; skew 500 ppm
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:10.561183 14127 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:10.562134 14127 webserver.cc:460] Webserver started at http://127.12.234.195:41959/ using document root <none> and password file <none>
I20260619 18:47:10.562772 14127 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:10.562842 14127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:10.563053 14127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:10.565229 14127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "fe1925b66b3f4bac8fc4cd69219f92af"
format_stamp: "Formatted at 2026-06-19 18:47:10 on dist-test-slave-fljv"
server_key: "318b4e8bb28932e6674e63e8d7919dd0"
server_key_iv: "2a28bddc2a3cebab5c15ec8a71a33457"
server_key_version: "encryptionkey@0"
I20260619 18:47:10.565786 14127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "fe1925b66b3f4bac8fc4cd69219f92af"
format_stamp: "Formatted at 2026-06-19 18:47:10 on dist-test-slave-fljv"
server_key: "318b4e8bb28932e6674e63e8d7919dd0"
server_key_iv: "2a28bddc2a3cebab5c15ec8a71a33457"
server_key_version: "encryptionkey@0"
I20260619 18:47:10.569614 14127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 18:47:10.572263 14142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:10.573560 14127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:10.573690 14127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "fe1925b66b3f4bac8fc4cd69219f92af"
format_stamp: "Formatted at 2026-06-19 18:47:10 on dist-test-slave-fljv"
server_key: "318b4e8bb28932e6674e63e8d7919dd0"
server_key_iv: "2a28bddc2a3cebab5c15ec8a71a33457"
server_key_version: "encryptionkey@0"
I20260619 18:47:10.573801 14127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:10.594480 14127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:10.597595 14127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:10.597787 14127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:10.598441 14127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:10.599336 14127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:10.599385 14127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:10.599438 14127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:10.599465 14127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:10.608018 14127 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:40241
I20260619 18:47:10.608047 14255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:40241 every 8 connection(s)
I20260619 18:47:10.609053 14127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:10.617899 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 14127
I20260619 18:47:10.618031 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:10.618326 13227 external_mini_cluster.cc:1428] Setting key 1ba164a198a318cc4d6449c2fdbbb7fa
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:10.623476 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.610966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:40797 (local address 127.12.234.254:45847)
0619 18:47:10.611087 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.611090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.612021 (+ 931us) server_negotiation.cc:407] Connection header received
0619 18:47:10.613493 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.613498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.613562 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.613646 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:10.615149 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.615684 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.616862 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.617099 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.617201 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.620074 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:10.620079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:10.620082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:10.620110 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:10.621486 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.621953 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.621955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.621956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.622024 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.622349 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.622352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.622353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.622536 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:10.622610 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.623235 (+ 625us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.623352 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:47:10.624303 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.611359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:45847 (local address 127.12.234.195:40797)
0619 18:47:10.611843 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:10.611891 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:10.613247 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:10.613854 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:10.613862 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:10.614396 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:10.614991 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.615006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.615816 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.615820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.616734 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.616743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.617258 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.617262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.617448 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.618237 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:10.618254 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:10.619916 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:10.621587 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:10.621593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:10.621607 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:10.621879 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:10.622084 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:10.622086 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:10.622087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:10.622270 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:10.622656 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:10.622663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:10.622928 (+ 265us) client_negotiation.cc:769] Sending connection context
0619 18:47:10.623096 (+ 168us) client_negotiation.cc:240] Negotiation successful
0619 18:47:10.623409 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":123,"threads_started":1}
I20260619 18:47:10.625408 14256 heartbeater.cc:347] Connected to a master server at 127.12.234.254:45847
I20260619 18:47:10.625643 14256 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:10.626157 14256 heartbeater.cc:511] Master 127.12.234.254:45847 requested a full tablet report, sending...
I20260619 18:47:10.627206 13797 ts_manager.cc:194] Registered new tserver with Master: fe1925b66b3f4bac8fc4cd69219f92af (127.12.234.195:40241)
I20260619 18:47:10.627769 13797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:40797
I20260619 18:47:10.632355 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:47:10.643384 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.636027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57188 (local address 127.12.234.254:45847)
0619 18:47:10.636161 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.636166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.636358 (+ 192us) server_negotiation.cc:407] Connection header received
0619 18:47:10.636513 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.636516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.636577 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.636645 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:10.637390 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.637912 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.638801 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.639015 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.639102 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.640417 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:10.640420 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:10.640422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:10.640447 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:10.641758 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.642155 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.642159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.642160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.642278 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.642524 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.642527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.642529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.642762 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:10.642878 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.643111 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.643223 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:47:10.648809 13797 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:10.651904 13797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:10.666640 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.661322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48108 (local address 127.12.234.194:46773)
0619 18:47:10.661968 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.661979 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.662001 (+ 22us) server_negotiation.cc:407] Connection header received
0619 18:47:10.662072 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.662077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.662318 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.662459 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:10.662849 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.663851 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.665106 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.665922 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.666048 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.666218 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.666390 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.666484 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":144,"threads_started":1}
I20260619 18:47:10.666642 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.660692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:46773 (local address 127.0.0.1:48108)
0619 18:47:10.661530 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:10.661542 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:10.661621 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:10.662612 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:10.662614 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:10.662629 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:10.662717 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.662723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.663966 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.663969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.664977 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.664986 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.666027 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.666029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.666138 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.666204 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 18:47:10.666351 (+ 147us) client_negotiation.cc:240] Negotiation successful
0619 18:47:10.666479 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":44,"thread_start_us":99,"threads_started":1}
I20260619 18:47:10.667037 14272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.661372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49850 (local address 127.12.234.195:40241)
0619 18:47:10.661791 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.661803 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.661961 (+ 158us) server_negotiation.cc:407] Connection header received
0619 18:47:10.662035 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.662040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.662259 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.662380 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:10.662734 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.663483 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.665346 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.666202 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.666311 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.666430 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.666707 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.666902 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":89,"threads_started":1}
I20260619 18:47:10.666643 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.661208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:40241 (local address 127.0.0.1:49850)
0619 18:47:10.661775 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:10.661794 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:10.661876 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:10.662512 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:10.662515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:10.662531 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:10.662614 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.662621 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.663605 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.663608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.665234 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.665241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.666302 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.666306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.666398 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.666419 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 18:47:10.666478 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 18:47:10.666532 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":159,"thread_start_us":118,"threads_started":1}
I20260619 18:47:10.667922 14267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.660393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:37185 (local address 127.0.0.1:51126)
0619 18:47:10.660798 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:10.660854 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:10.660986 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:10.661942 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:10.661948 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:10.661968 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:10.662116 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.662122 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.663720 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.663723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.665216 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.665227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.667548 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.667552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.667638 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.667658 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 18:47:10.667728 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 18:47:10.667797 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":124,"thread_start_us":99,"threads_started":1}
I20260619 18:47:10.668519 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.660524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.12.234.193:37185)
0619 18:47:10.661407 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.661426 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.661453 (+ 27us) server_negotiation.cc:407] Connection header received
0619 18:47:10.661530 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.661536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.661773 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.662296 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:10.662451 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.663592 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.666485 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.667436 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.667551 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.667668 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.668288 (+ 620us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.668384 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":160,"threads_started":1}
I20260619 18:47:10.669968 14056 tablet_service.cc:1467] Processing CreateTablet for tablet d5c3f45bc7f94ccb981a975ba6118b5e (DEFAULT_TABLE table=test-table [id=846e29e992604dee9604b933ff40f7dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:10.669970 14190 tablet_service.cc:1467] Processing CreateTablet for tablet d5c3f45bc7f94ccb981a975ba6118b5e (DEFAULT_TABLE table=test-table [id=846e29e992604dee9604b933ff40f7dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:10.671186 14056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5c3f45bc7f94ccb981a975ba6118b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:10.671190 14190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5c3f45bc7f94ccb981a975ba6118b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:10.676971 13922 tablet_service.cc:1467] Processing CreateTablet for tablet d5c3f45bc7f94ccb981a975ba6118b5e (DEFAULT_TABLE table=test-table [id=846e29e992604dee9604b933ff40f7dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:10.678004 13922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5c3f45bc7f94ccb981a975ba6118b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:10.679050 14273 tablet_bootstrap.cc:492] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af: Bootstrap starting.
I20260619 18:47:10.680442 14274 tablet_bootstrap.cc:492] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35: Bootstrap starting.
I20260619 18:47:10.681444 14273 tablet_bootstrap.cc:654] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:10.682641 14274 tablet_bootstrap.cc:654] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:10.682803 14273 log.cc:824] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:10.683430 14274 log.cc:824] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:10.684973 14273 tablet_bootstrap.cc:492] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af: No bootstrap required, opened a new log
I20260619 18:47:10.685168 14274 tablet_bootstrap.cc:492] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35: No bootstrap required, opened a new log
I20260619 18:47:10.685170 14273 ts_tablet_manager.cc:1397] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 18:47:10.685366 14274 ts_tablet_manager.cc:1397] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260619 18:47:10.685463 14277 tablet_bootstrap.cc:492] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794: Bootstrap starting.
I20260619 18:47:10.687790 14277 tablet_bootstrap.cc:654] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:10.688213 14273 raft_consensus.cc:348] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } }
I20260619 18:47:10.688457 14273 raft_consensus.cc:374] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:10.688508 14273 raft_consensus.cc:729] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe1925b66b3f4bac8fc4cd69219f92af, State: Initialized, Role: FOLLOWER
I20260619 18:47:10.688632 14277 log.cc:824] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:10.688935 14273 consensus_queue.cc:260] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } }
I20260619 18:47:10.689075 14274 raft_consensus.cc:348] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } }
I20260619 18:47:10.689358 14274 raft_consensus.cc:374] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:10.689425 14274 raft_consensus.cc:729] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea86eb6cdfa4b0f87c8ff38229f4f35, State: Initialized, Role: FOLLOWER
I20260619 18:47:10.689873 14274 consensus_queue.cc:260] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } }
I20260619 18:47:10.689950 14273 ts_tablet_manager.cc:1428] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 18:47:10.690339 14256 heartbeater.cc:503] Master 127.12.234.254:45847 was elected leader, sending a full tablet report...
I20260619 18:47:10.690562 14122 heartbeater.cc:503] Master 127.12.234.254:45847 was elected leader, sending a full tablet report...
I20260619 18:47:10.690796 14274 ts_tablet_manager.cc:1428] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260619 18:47:10.692121 14277 tablet_bootstrap.cc:492] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794: No bootstrap required, opened a new log
I20260619 18:47:10.692320 14277 ts_tablet_manager.cc:1397] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 18:47:10.695974 14277 raft_consensus.cc:348] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } }
I20260619 18:47:10.696220 14277 raft_consensus.cc:374] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:10.696292 14277 raft_consensus.cc:729] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b64d1a3c85394a9e9bde0a327c217794, State: Initialized, Role: FOLLOWER
I20260619 18:47:10.696691 14277 consensus_queue.cc:260] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } }
I20260619 18:47:10.697329 13988 heartbeater.cc:503] Master 127.12.234.254:45847 was elected leader, sending a full tablet report...
I20260619 18:47:10.697568 14277 ts_tablet_manager.cc:1428] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 18:47:10.728996 14281 raft_consensus.cc:482] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:10.729234 14281 raft_consensus.cc:504] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } }
I20260619 18:47:10.730424 14281 leader_election.cc:288] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe1925b66b3f4bac8fc4cd69219f92af (127.12.234.195:40241), cea86eb6cdfa4b0f87c8ff38229f4f35 (127.12.234.194:46773)
I20260619 18:47:10.735548 14272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.730824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:47267 (local address 127.12.234.195:40241)
0619 18:47:10.730957 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.730962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.731242 (+ 280us) server_negotiation.cc:407] Connection header received
0619 18:47:10.731390 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.731393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.731443 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.731530 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:10.732009 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.732829 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.734587 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.734991 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.735107 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.735165 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.735340 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.735416 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 18:47:10.735556 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.730901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:59121 (local address 127.12.234.194:46773)
0619 18:47:10.731108 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.731112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.731290 (+ 178us) server_negotiation.cc:407] Connection header received
0619 18:47:10.731389 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.731392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.731445 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.731531 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:10.732009 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.732829 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.734541 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.735006 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.735122 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.735173 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.735367 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.735436 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260619 18:47:10.735598 14282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.730731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:46773 (local address 127.12.234.193:59121)
0619 18:47:10.731210 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:10.731222 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:10.731298 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:10.731547 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:10.731550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:10.731591 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:10.731843 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.731851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.732996 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.732999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.734409 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.734420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.735124 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.735127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.735253 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.735271 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 18:47:10.735342 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 18:47:10.735416 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":164,"spinlock_wait_cycles":4992,"thread_start_us":110,"threads_started":1}
I20260619 18:47:10.735603 14283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.730731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:40241 (local address 127.12.234.193:47267)
0619 18:47:10.731125 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:10.731141 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:10.731286 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:10.731547 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:10.731550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:10.731589 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:10.731843 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.731851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.732975 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.732979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.734444 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:10.734456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.735108 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.735112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:10.735210 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.735248 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 18:47:10.735304 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 18:47:10.735381 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":198,"spinlock_wait_cycles":33792,"thread_start_us":133,"threads_started":1}
I20260619 18:47:10.736333 14210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5c3f45bc7f94ccb981a975ba6118b5e" candidate_uuid: "b64d1a3c85394a9e9bde0a327c217794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" is_pre_election: true
I20260619 18:47:10.736423 14076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5c3f45bc7f94ccb981a975ba6118b5e" candidate_uuid: "b64d1a3c85394a9e9bde0a327c217794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" is_pre_election: true
I20260619 18:47:10.736657 14210 raft_consensus.cc:2454] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b64d1a3c85394a9e9bde0a327c217794 in term 0.
I20260619 18:47:10.736689 14076 raft_consensus.cc:2454] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b64d1a3c85394a9e9bde0a327c217794 in term 0.
I20260619 18:47:10.737094 13877 leader_election.cc:302] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b64d1a3c85394a9e9bde0a327c217794, fe1925b66b3f4bac8fc4cd69219f92af; no voters:
I20260619 18:47:10.737360 14281 raft_consensus.cc:2781] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:10.737452 14281 raft_consensus.cc:482] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:10.737483 14281 raft_consensus.cc:3037] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:10.738680 14281 raft_consensus.cc:504] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } }
I20260619 18:47:10.738998 14281 leader_election.cc:288] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [CANDIDATE]: Term 1 election: Requested vote from peers fe1925b66b3f4bac8fc4cd69219f92af (127.12.234.195:40241), cea86eb6cdfa4b0f87c8ff38229f4f35 (127.12.234.194:46773)
I20260619 18:47:10.739348 14076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5c3f45bc7f94ccb981a975ba6118b5e" candidate_uuid: "b64d1a3c85394a9e9bde0a327c217794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35"
I20260619 18:47:10.739482 14076 raft_consensus.cc:3037] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:10.739555 14210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5c3f45bc7f94ccb981a975ba6118b5e" candidate_uuid: "b64d1a3c85394a9e9bde0a327c217794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe1925b66b3f4bac8fc4cd69219f92af"
I20260619 18:47:10.739719 14210 raft_consensus.cc:3037] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:10.740581 14076 raft_consensus.cc:2454] T d5c3f45bc7f94ccb981a975ba6118b5e P cea86eb6cdfa4b0f87c8ff38229f4f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b64d1a3c85394a9e9bde0a327c217794 in term 1.
I20260619 18:47:10.740809 14210 raft_consensus.cc:2454] T d5c3f45bc7f94ccb981a975ba6118b5e P fe1925b66b3f4bac8fc4cd69219f92af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b64d1a3c85394a9e9bde0a327c217794 in term 1.
I20260619 18:47:10.740880 13875 leader_election.cc:302] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b64d1a3c85394a9e9bde0a327c217794, cea86eb6cdfa4b0f87c8ff38229f4f35; no voters:
I20260619 18:47:10.741076 14281 raft_consensus.cc:2781] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:10.741299 14281 raft_consensus.cc:686] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [term 1 LEADER]: Becoming Leader. State: Replica: b64d1a3c85394a9e9bde0a327c217794, State: Running, Role: LEADER
I20260619 18:47:10.741588 14281 consensus_queue.cc:237] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } }
I20260619 18:47:10.744889 13797 catalog_manager.cc:5495] T d5c3f45bc7f94ccb981a975ba6118b5e P b64d1a3c85394a9e9bde0a327c217794 reported cstate change: term changed from 0 to 1, leader changed from <none> to b64d1a3c85394a9e9bde0a327c217794 (127.12.234.193). New cstate: current_term: 1 leader_uuid: "b64d1a3c85394a9e9bde0a327c217794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b64d1a3c85394a9e9bde0a327c217794" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe1925b66b3f4bac8fc4cd69219f92af" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 40241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea86eb6cdfa4b0f87c8ff38229f4f35" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 46773 } health_report { overall_health: UNKNOWN } } }
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:10.768195 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.759291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51140 (local address 127.12.234.193:37185)
0619 18:47:10.759463 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.759468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.759586 (+ 118us) server_negotiation.cc:407] Connection header received
0619 18:47:10.759801 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.759805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.759852 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.759929 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:10.760675 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.761187 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.762070 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.762339 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.762419 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.764942 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:10.764954 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:10.764970 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:10.764995 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:10.766481 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.766945 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.766951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.766952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.767052 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.767350 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.767352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.767354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.767625 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:10.767816 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.767954 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.768056 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:10.793802 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.785493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51154 (local address 127.12.234.193:37185)
0619 18:47:10.785670 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.785676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.785831 (+ 155us) server_negotiation.cc:407] Connection header received
0619 18:47:10.785976 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.785979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.786029 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.786099 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:10.786832 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.787389 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.788199 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.788426 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.788512 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.790920 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:10.790924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:10.790926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:10.790954 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:10.792220 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.792612 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.792615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.792617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.792693 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.792929 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.792932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.792934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.793106 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:10.793283 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.793525 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.793642 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[13752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:10.819684 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:10.810946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.12.234.193:37185)
0619 18:47:10.811116 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:10.811120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:10.811279 (+ 159us) server_negotiation.cc:407] Connection header received
0619 18:47:10.811394 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:10.811397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:10.811443 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:10.811517 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:10.812247 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.812777 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.813720 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:10.813969 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:10.814050 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:10.816817 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:10.816822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:10.816824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:10.816853 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:10.818229 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.818587 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.818589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.818590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.818657 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:10.818884 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:10.818887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:10.818889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:10.819071 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:10.819276 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:10.819427 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 18:47:10.819521 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 18:47:10.821064 13902 tablet_service.cc:2908] Rejecting scan request for tablet d5c3f45bc7f94ccb981a975ba6118b5e: Uninitialized: clean time has not yet been initialized
I20260619 18:47:10.823084 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 13859
I20260619 18:47:10.828291 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 13993
I20260619 18:47:10.832845 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 14127
I20260619 18:47:10.839301 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 13765
2026-06-19T18:47:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2260 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[14298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[14298](info): set up 2 sockets
Jun 19 18:47:10 dist-test-slave-fljv krb5kdc[14298](info): commencing operation
krb5kdc: starting...
W20260619 18:47:11.793746 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:11 dist-test-slave-fljv krb5kdc[14298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:11.917362 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40769
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:40769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:12.035616 14311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:12.035954 14311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:12.036000 14311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:12.040289 14311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:12.040373 14311 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 18:47:12.040390 14311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:12.040406 14311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:12.040422 14311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:12.040436 14311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:12.047192 14311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:40769
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:12.048306 14311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:12.049363 14311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:12.058250 14316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:12.058252 14317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:12.058444 14311 server_base.cc:1034] running on GCE node
W20260619 18:47:12.058252 14319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:12.059141 14311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:12.060343 14311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:12.061507 14311 hybrid_clock.cc:648] HybridClock initialized: now 1781894832061492 us; error 33 us; skew 500 ppm
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:12.064280 14311 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:12.065217 14311 webserver.cc:460] Webserver started at http://127.12.234.254:41511/ using document root <none> and password file <none>
I20260619 18:47:12.065761 14311 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:12.065819 14311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:12.065986 14311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:12.068130 14311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "e179312d10164a3d841c2c99734624dd"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "b4d2754b55770c9bfc71e192acf05be0"
server_key_iv: "1f91ebc158abd0233e7481f27c49fee7"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.068675 14311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "e179312d10164a3d841c2c99734624dd"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "b4d2754b55770c9bfc71e192acf05be0"
server_key_iv: "1f91ebc158abd0233e7481f27c49fee7"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.072566 14311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:12.075870 14326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:12.077636 14311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:12.077790 14311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "e179312d10164a3d841c2c99734624dd"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "b4d2754b55770c9bfc71e192acf05be0"
server_key_iv: "1f91ebc158abd0233e7481f27c49fee7"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.077919 14311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:12.099519 14311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:12.102857 14311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:12.103041 14311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:12.110900 14311 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:40769
I20260619 18:47:12.110930 14388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:40769 every 8 connection(s)
I20260619 18:47:12.111882 14311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:12.114069 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 14311
I20260619 18:47:12.114249 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:12.114495 13227 external_mini_cluster.cc:1428] Setting key 9ef85f617f5d26b1d65bcbb886da71ca
I20260619 18:47:12.114928 14389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:12.121369 14389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd: Bootstrap starting.
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:12.123778 14389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:12.124986 14389 log.cc:824] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:12.127043 14389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd: No bootstrap required, opened a new log
I20260619 18:47:12.128314 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.115923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.12.234.254:40769)
0619 18:47:12.116393 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.116409 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.116449 (+ 40us) server_negotiation.cc:407] Connection header received
0619 18:47:12.117142 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.117162 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.117515 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.117873 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:12.118717 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.119731 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.120576 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.121075 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.121217 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.123653 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:12.123658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:12.123673 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:12.123706 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:12.125518 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.125951 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.125956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.125960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.126076 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.126358 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.126361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.126363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.126789 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:12.126968 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.127252 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.127529 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":142,"threads_started":1}
I20260619 18:47:12.130241 14389 raft_consensus.cc:348] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e179312d10164a3d841c2c99734624dd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40769 } }
I20260619 18:47:12.130548 14389 raft_consensus.cc:374] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:12.130613 14389 raft_consensus.cc:729] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e179312d10164a3d841c2c99734624dd, State: Initialized, Role: FOLLOWER
I20260619 18:47:12.131004 14389 consensus_queue.cc:260] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e179312d10164a3d841c2c99734624dd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40769 } }
I20260619 18:47:12.131135 14389 raft_consensus.cc:388] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:12.131174 14389 raft_consensus.cc:482] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:12.131243 14389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:12.132138 14389 raft_consensus.cc:504] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e179312d10164a3d841c2c99734624dd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40769 } }
I20260619 18:47:12.132445 14389 leader_election.cc:302] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e179312d10164a3d841c2c99734624dd; no voters:
I20260619 18:47:12.132728 14389 leader_election.cc:288] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:12.132862 14394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:12.133145 14394 raft_consensus.cc:686] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [term 1 LEADER]: Becoming Leader. State: Replica: e179312d10164a3d841c2c99734624dd, State: Running, Role: LEADER
I20260619 18:47:12.133448 14394 consensus_queue.cc:237] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e179312d10164a3d841c2c99734624dd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40769 } }
I20260619 18:47:12.133980 14389 sys_catalog.cc:564] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:12.135154 14395 sys_catalog.cc:455] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e179312d10164a3d841c2c99734624dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e179312d10164a3d841c2c99734624dd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40769 } } }
I20260619 18:47:12.135126 14396 sys_catalog.cc:455] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e179312d10164a3d841c2c99734624dd. Latest consensus state: current_term: 1 leader_uuid: "e179312d10164a3d841c2c99734624dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e179312d10164a3d841c2c99734624dd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40769 } } }
I20260619 18:47:12.135268 14395 sys_catalog.cc:458] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:12.135268 14396 sys_catalog.cc:458] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:12.135656 14403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:12.139467 14403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:12.145449 14403 catalog_manager.cc:1347] Generated new cluster ID: afaa4b131fee4d6caac53bee0f4fa9a7
I20260619 18:47:12.145537 14403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:12.171576 14403 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:12.173074 14403 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:12.188992 14403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd: Generated new TSK 0
I20260619 18:47:12.189947 14403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:12.229737 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40769
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:12.344118 14415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:12.344410 14415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:12.344457 14415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:12.348819 14415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:12.348898 14415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:12.348939 14415 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 18:47:12.348978 14415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:12.354545 14415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.254:40769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:12.355706 14415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:12.356770 14415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:47:12.367014 14415 server_base.cc:1034] running on GCE node
W20260619 18:47:12.366957 14421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:12.366951 14420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:12.366953 14423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:12.367722 14415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:12.368423 14415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:12.369632 14415 hybrid_clock.cc:648] HybridClock initialized: now 1781894832369600 us; error 46 us; skew 500 ppm
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:12.372758 14415 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:12.373700 14415 webserver.cc:460] Webserver started at http://127.12.234.193:33273/ using document root <none> and password file <none>
I20260619 18:47:12.374318 14415 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:12.374387 14415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:12.374574 14415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:12.376667 14415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "6cacb395492042528e0c14e0e5b3e593"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "dc9aa5ac60e9971f56a23071d57e2dbf"
server_key_iv: "ec1dffe0870c6fd64600d4c95ef3131d"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.377192 14415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "6cacb395492042528e0c14e0e5b3e593"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "dc9aa5ac60e9971f56a23071d57e2dbf"
server_key_iv: "ec1dffe0870c6fd64600d4c95ef3131d"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.381033 14415 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:12.383795 14430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:12.385179 14415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:12.385308 14415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "6cacb395492042528e0c14e0e5b3e593"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "dc9aa5ac60e9971f56a23071d57e2dbf"
server_key_iv: "ec1dffe0870c6fd64600d4c95ef3131d"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.385432 14415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:12.405236 14415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:12.408561 14415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:12.408739 14415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:12.410272 14415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:12.410322 14415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:12.410364 14415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:12.410392 14415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:12.420805 14415 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:34489
I20260619 18:47:12.421831 14415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:12.422345 14545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:34489 every 8 connection(s)
I20260619 18:47:12.424803 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.411192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:55027 (local address 127.12.234.254:40769)
0619 18:47:12.411335 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.411339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.412327 (+ 988us) server_negotiation.cc:407] Connection header received
0619 18:47:12.413826 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.413830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.413894 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.413993 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:12.415611 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.416285 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.417990 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.418287 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.418372 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.421457 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:12.421460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:12.421462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:12.421489 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:12.423045 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.423531 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.423534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.423535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.423612 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.423882 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.423884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.423885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.424057 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:12.424128 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.424560 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.424682 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 18:47:12.425683 14439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.411650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40769 (local address 127.12.234.193:55027)
0619 18:47:12.412145 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:12.412204 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:12.413559 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:12.414210 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:12.414219 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:12.414710 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:12.415439 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.415460 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.416393 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.416397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.417868 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.417878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.418441 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.418446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.418668 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.419338 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:12.419355 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:12.421262 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:12.423152 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.423156 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.423171 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.423445 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.423682 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.423684 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.423685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.423813 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.424128 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:12.424134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:12.424363 (+ 229us) client_negotiation.cc:769] Sending connection context
0619 18:47:12.424515 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 18:47:12.424783 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
I20260619 18:47:12.426301 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 14415
I20260619 18:47:12.426410 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:12.426692 13227 external_mini_cluster.cc:1428] Setting key f6b08f864ac3bd357c881a5bff540795
I20260619 18:47:12.426997 14546 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40769
I20260619 18:47:12.427240 14546 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:12.427811 14546 heartbeater.cc:511] Master 127.12.234.254:40769 requested a full tablet report, sending...
I20260619 18:47:12.429399 14342 ts_manager.cc:194] Registered new tserver with Master: 6cacb395492042528e0c14e0e5b3e593 (127.12.234.193:34489)
I20260619 18:47:12.430924 14342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:55027
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:12.436232 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.429367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:34647 (local address 127.12.234.254:40769)
0619 18:47:12.429477 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.429481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.429778 (+ 297us) server_negotiation.cc:407] Connection header received
0619 18:47:12.429950 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.429954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.429996 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.430036 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:12.430583 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.431095 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.432015 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.432227 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.432310 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.433347 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:12.433351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:12.433354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:12.433378 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:12.434697 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.435141 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.435144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.435145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.435217 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.435462 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.435466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.435468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.435647 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:12.435728 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.435964 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.436079 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 18:47:12.436256 14552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.429294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40769 (local address 127.12.234.193:34647)
0619 18:47:12.429697 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:12.429711 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:12.429850 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:12.430156 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:12.430158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:12.430374 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:12.430474 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.430479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.431221 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.431224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.431917 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.431925 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.432319 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.432323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.432410 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.432878 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:12.432890 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:12.433225 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:12.434793 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.434796 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.434799 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.435052 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.435309 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.435311 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.435312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.435383 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.435730 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:12.435733 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:12.435817 (+ 84us) client_negotiation.cc:769] Sending connection context
0619 18:47:12.435904 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 18:47:12.436069 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":228,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:12.454383 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40769
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:12.567179 14555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:12.567467 14555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:12.567533 14555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:12.571794 14555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:12.571864 14555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:12.571920 14555 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 18:47:12.571974 14555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:12.577545 14555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.254:40769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:12.578784 14555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:12.579852 14555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:12.589527 14561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:12.589542 14563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:12.589507 14560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:12.589859 14555 server_base.cc:1034] running on GCE node
I20260619 18:47:12.590433 14555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:12.591105 14555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:12.592298 14555 hybrid_clock.cc:648] HybridClock initialized: now 1781894832592274 us; error 36 us; skew 500 ppm
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:12.595140 14555 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:12.596052 14555 webserver.cc:460] Webserver started at http://127.12.234.194:40331/ using document root <none> and password file <none>
I20260619 18:47:12.596606 14555 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:12.596666 14555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:12.596866 14555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:12.599036 14555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "f4d18e5b23a542548383da12f1f8c8c2"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "9a2ae47fb0546055d946b49f08ff4e6f"
server_key_iv: "6e51e52126780459fb6c2d83028c4fdb"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.599577 14555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "f4d18e5b23a542548383da12f1f8c8c2"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "9a2ae47fb0546055d946b49f08ff4e6f"
server_key_iv: "6e51e52126780459fb6c2d83028c4fdb"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.603395 14555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 18:47:12.606271 14570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:12.607733 14555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 18:47:12.607872 14555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "f4d18e5b23a542548383da12f1f8c8c2"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "9a2ae47fb0546055d946b49f08ff4e6f"
server_key_iv: "6e51e52126780459fb6c2d83028c4fdb"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.607988 14555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:12.635066 14555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:12.638906 14555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:12.639106 14555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:12.640801 14555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:12.640858 14555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:12.640913 14555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:12.640941 14555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:12.651345 14555 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:35581
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:12.651518 14685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:35581 every 8 connection(s)
I20260619 18:47:12.652334 14555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:12.655540 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.641701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:56103 (local address 127.12.234.254:40769)
0619 18:47:12.641848 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.641853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.642867 (+ 1014us) server_negotiation.cc:407] Connection header received
0619 18:47:12.644142 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.644146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.644225 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.644330 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:12.646130 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.646963 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.648793 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.649016 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.649117 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.652095 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:12.652098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:12.652100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:12.652130 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:12.653615 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.654081 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.654083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.654085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.654156 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.654546 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.654548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.654549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.654733 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:12.654805 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.655318 (+ 513us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.655421 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 18:47:12.656356 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.642127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40769 (local address 127.12.234.194:56103)
0619 18:47:12.642667 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:12.642739 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:12.643869 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:12.644551 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:12.644561 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:12.645095 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:12.645896 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.645923 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.647094 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.647098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.648631 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.648642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.649192 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.649196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.649433 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.650208 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:12.650234 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:12.651894 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:12.653733 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.653738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.653750 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.653998 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.654331 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.654333 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.654335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.654474 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.654806 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:12.654811 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:12.655099 (+ 288us) client_negotiation.cc:769] Sending connection context
0619 18:47:12.655261 (+ 162us) client_negotiation.cc:240] Negotiation successful
0619 18:47:12.655516 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":110,"threads_started":1}
I20260619 18:47:12.657615 14686 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40769
I20260619 18:47:12.657763 14686 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:12.658316 14686 heartbeater.cc:511] Master 127.12.234.254:40769 requested a full tablet report, sending...
I20260619 18:47:12.659354 14343 ts_manager.cc:194] Registered new tserver with Master: f4d18e5b23a542548383da12f1f8c8c2 (127.12.234.194:35581)
I20260619 18:47:12.660064 14343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:56103
I20260619 18:47:12.661332 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 14555
I20260619 18:47:12.661441 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:12.661728 13227 external_mini_cluster.cc:1428] Setting key b000ce559a7e4a7ff36c9eb522d56445
I20260619 18:47:12.667472 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.659828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:50375 (local address 127.12.234.254:40769)
0619 18:47:12.659944 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.659948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.660127 (+ 179us) server_negotiation.cc:407] Connection header received
0619 18:47:12.660240 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.660243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.660284 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.660356 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:12.661173 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.661886 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.662771 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.663022 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.663129 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.664216 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:12.664229 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:12.664231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:12.664257 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:12.665794 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.666293 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.666297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.666299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.666381 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.666619 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.666623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.666625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.666858 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:12.666943 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.667213 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.667317 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 18:47:12.667483 14692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.659732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40769 (local address 127.12.234.194:50375)
0619 18:47:12.660049 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:12.660061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:12.660143 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:12.660610 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:12.660613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:12.660842 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:12.660961 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.660968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.662012 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.662015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.662673 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.662682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.663164 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.663168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.663258 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.663759 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:12.663771 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:12.664084 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:12.665931 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.665935 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.665937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.666172 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.666459 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.666462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.666464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.666542 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.666950 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:12.666953 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:12.667044 (+ 91us) client_negotiation.cc:769] Sending connection context
0619 18:47:12.667146 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 18:47:12.667303 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:12.688898 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40769
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:12.801996 14695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:12.802307 14695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:12.802356 14695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:12.806421 14695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:12.806491 14695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:12.806533 14695 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 18:47:12.806577 14695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:12.812158 14695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.254:40769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:12.813284 14695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:12.814383 14695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:12.824263 14703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:12.824277 14701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:12.824263 14700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:12.824587 14695 server_base.cc:1034] running on GCE node
I20260619 18:47:12.825048 14695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:12.825798 14695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:12.826979 14695 hybrid_clock.cc:648] HybridClock initialized: now 1781894832826944 us; error 47 us; skew 500 ppm
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:12.830073 14695 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:12.831180 14695 webserver.cc:460] Webserver started at http://127.12.234.195:37371/ using document root <none> and password file <none>
I20260619 18:47:12.831769 14695 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:12.831843 14695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:12.832074 14695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:12.834226 14695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "d542966cde50417fb3fa9bfa5e78e732"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "0e78c192771bd3d8564bc029d73c669e"
server_key_iv: "d6791f76c4b0da48bf7fe4082079faa8"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.834828 14695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "d542966cde50417fb3fa9bfa5e78e732"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "0e78c192771bd3d8564bc029d73c669e"
server_key_iv: "d6791f76c4b0da48bf7fe4082079faa8"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.838801 14695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:47:12.841409 14710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:12.842693 14695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:12.842793 14695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "d542966cde50417fb3fa9bfa5e78e732"
format_stamp: "Formatted at 2026-06-19 18:47:12 on dist-test-slave-fljv"
server_key: "0e78c192771bd3d8564bc029d73c669e"
server_key_iv: "d6791f76c4b0da48bf7fe4082079faa8"
server_key_version: "encryptionkey@0"
I20260619 18:47:12.842881 14695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:12.866371 14695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:12.869581 14695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:12.869740 14695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:12.871266 14695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:12.871327 14695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:12.871368 14695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:12.871382 14695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:12.881372 14695 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:44991
I20260619 18:47:12.881431 14825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:44991 every 8 connection(s)
I20260619 18:47:12.882473 14695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:12.884639 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.872085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:57793 (local address 127.12.234.254:40769)
0619 18:47:12.872206 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.872210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.873325 (+ 1115us) server_negotiation.cc:407] Connection header received
0619 18:47:12.874421 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.874425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.874493 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.874602 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:12.875955 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.876781 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.878058 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.878418 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.878524 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.880978 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:12.880982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:12.880985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:12.881018 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:12.882707 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.883252 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.883256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.883258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.883347 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.883664 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.883668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.883669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.883867 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:12.883956 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.884401 (+ 445us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.884503 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:47:12.885533 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.872673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40769 (local address 127.12.234.195:57793)
0619 18:47:12.873169 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:12.873214 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:12.874131 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:12.874801 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:12.874812 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:12.875241 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:12.875825 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.875838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.876904 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.876907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.877924 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.877936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.878582 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.878587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.878756 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.879347 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:12.879363 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:12.880794 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:12.882807 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.882813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.882828 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.883167 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.883427 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.883429 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.883430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.883555 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.883956 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:12.883961 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:12.884194 (+ 233us) client_negotiation.cc:769] Sending connection context
0619 18:47:12.884349 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 18:47:12.884593 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":138,"threads_started":1}
I20260619 18:47:12.885547 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 14695
I20260619 18:47:12.885646 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:12.885926 13227 external_mini_cluster.cc:1428] Setting key 2452ebb85d31f9f27c61ea03fd164cb4
I20260619 18:47:12.886999 14826 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40769
I20260619 18:47:12.887156 14826 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:12.887662 14826 heartbeater.cc:511] Master 127.12.234.254:40769 requested a full tablet report, sending...
I20260619 18:47:12.888640 14342 ts_manager.cc:194] Registered new tserver with Master: d542966cde50417fb3fa9bfa5e78e732 (127.12.234.195:44991)
I20260619 18:47:12.889384 14342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:57793
I20260619 18:47:12.896021 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.889658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:51995 (local address 127.12.234.254:40769)
0619 18:47:12.889743 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.889746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.889756 (+ 10us) server_negotiation.cc:407] Connection header received
0619 18:47:12.889783 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.889785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.889818 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.889880 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:12.890408 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.890936 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.891976 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.892220 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.892298 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.893243 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:12.893246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:12.893248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:12.893269 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:12.894568 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.894934 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.894936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.894938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.895010 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.895296 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.895297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.895298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.895458 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:12.895533 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.895826 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.895910 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 18:47:12.896086 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.889122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40769 (local address 127.12.234.195:51995)
0619 18:47:12.889413 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:12.889427 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:12.889506 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:12.889917 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:12.889920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:12.890138 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:12.890279 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.890287 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.891053 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.891056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.891870 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.891879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.892308 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.892310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.892374 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.892788 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:12.892799 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:12.893105 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:12.894663 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.894665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.894667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.894850 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.895131 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:12.895134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:12.895135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:12.895220 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:12.895591 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:12.895594 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:12.895681 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 18:47:12.895789 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 18:47:12.895902 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":80,"threads_started":1}
I20260619 18:47:12.899608 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 18:47:12 dist-test-slave-fljv krb5kdc[14298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:12.923995 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.916103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.12.234.254:40769)
0619 18:47:12.916259 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.916262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.916276 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:12.916316 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.916319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.916358 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.916425 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:12.917036 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.917537 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.918488 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.918711 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.918783 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.921235 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:12.921239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:12.921241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:12.921266 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:12.922478 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.922854 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.922856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.922857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.922924 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:12.923150 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:12.923153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:12.923154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:12.923366 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:12.923474 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.923760 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.923876 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 18:47:12.926714 14342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:12.929791 14342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:12.944271 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.939801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.12.234.194:35581)
0619 18:47:12.940089 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.940100 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.940123 (+ 23us) server_negotiation.cc:407] Connection header received
0619 18:47:12.940207 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.940211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.940359 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.940502 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:12.940709 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.941571 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.942854 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.943633 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.943759 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.943896 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.944026 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.944115 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":106,"threads_started":1}
I20260619 18:47:12.944260 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.939080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:35581 (local address 127.0.0.1:40248)
0619 18:47:12.939750 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:12.939761 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:12.939831 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:12.940488 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:12.940491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:12.940508 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:12.940604 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.940610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.941695 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.941699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.942733 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.942742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.943741 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.943745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.943847 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.943881 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 18:47:12.943990 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 18:47:12.944115 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":70,"thread_start_us":87,"threads_started":1}
I20260619 18:47:12.944542 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.938461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45152 (local address 127.12.234.193:34489)
0619 18:47:12.938870 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.938878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.939121 (+ 243us) server_negotiation.cc:407] Connection header received
0619 18:47:12.939362 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.939369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.939591 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.939746 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:12.940229 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.941325 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.942916 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.943674 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.943782 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.943899 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.944324 (+ 425us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.944423 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":97,"threads_started":1}
I20260619 18:47:12.944679 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.938328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34489 (local address 127.0.0.1:45152)
0619 18:47:12.939012 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:12.939027 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:12.939197 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:12.939913 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:12.939919 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:12.939939 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:12.940089 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.940097 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.941429 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.941432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.942786 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.942797 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.943960 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.943963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.944070 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.944095 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 18:47:12.944155 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 18:47:12.944223 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":30,"thread_start_us":114,"threads_started":1}
I20260619 18:47:12.945261 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.938325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:44991 (local address 127.0.0.1:40950)
0619 18:47:12.938670 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:12.938719 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:12.938844 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:12.940745 (+ 1901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:12.940748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:12.940766 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:12.940858 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.940864 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.942061 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.942064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.943180 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:12.943189 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.944318 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.944894 (+ 576us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:12.944991 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.945013 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:47:12.945086 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 18:47:12.945147 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":19,"thread_start_us":93,"threads_started":1}
I20260619 18:47:12.945358 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:12.939627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.12.234.195:44991)
0619 18:47:12.940298 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:12.940310 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:12.940330 (+ 20us) server_negotiation.cc:407] Connection header received
0619 18:47:12.940414 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:12.940419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:12.940619 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:12.940770 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:12.940964 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.941949 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.943301 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:12.944040 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:12.944162 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:12.944265 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:12.945132 (+ 867us) server_negotiation.cc:299] Negotiation successful
0619 18:47:12.945225 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":107,"threads_started":1}
I20260619 18:47:12.947268 14620 tablet_service.cc:1467] Processing CreateTablet for tablet 569c99b5a67a4863ba7ac5cf623709cb (DEFAULT_TABLE table=test-table [id=94a825d8b5b544dc96bd5731da360f5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:12.947358 14480 tablet_service.cc:1467] Processing CreateTablet for tablet 569c99b5a67a4863ba7ac5cf623709cb (DEFAULT_TABLE table=test-table [id=94a825d8b5b544dc96bd5731da360f5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:12.948166 14480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 569c99b5a67a4863ba7ac5cf623709cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:12.948167 14620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 569c99b5a67a4863ba7ac5cf623709cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:12.952500 14760 tablet_service.cc:1467] Processing CreateTablet for tablet 569c99b5a67a4863ba7ac5cf623709cb (DEFAULT_TABLE table=test-table [id=94a825d8b5b544dc96bd5731da360f5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:12.953604 14760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 569c99b5a67a4863ba7ac5cf623709cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:12.954007 14849 tablet_bootstrap.cc:492] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: Bootstrap starting.
I20260619 18:47:12.956192 14850 tablet_bootstrap.cc:492] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: Bootstrap starting.
I20260619 18:47:12.956864 14849 tablet_bootstrap.cc:654] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:12.957628 14849 log.cc:824] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:12.958468 14850 tablet_bootstrap.cc:654] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:12.959359 14850 log.cc:824] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:12.959478 14849 tablet_bootstrap.cc:492] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: No bootstrap required, opened a new log
I20260619 18:47:12.959671 14849 ts_tablet_manager.cc:1397] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260619 18:47:12.961207 14853 tablet_bootstrap.cc:492] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: Bootstrap starting.
I20260619 18:47:12.963227 14853 tablet_bootstrap.cc:654] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:12.963397 14850 tablet_bootstrap.cc:492] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: No bootstrap required, opened a new log
I20260619 18:47:12.963615 14850 ts_tablet_manager.cc:1397] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260619 18:47:12.964030 14853 log.cc:824] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:12.964264 14849 raft_consensus.cc:348] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } }
I20260619 18:47:12.964560 14849 raft_consensus.cc:374] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:12.964619 14849 raft_consensus.cc:729] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cacb395492042528e0c14e0e5b3e593, State: Initialized, Role: FOLLOWER
I20260619 18:47:12.965073 14849 consensus_queue.cc:260] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } }
I20260619 18:47:12.965822 14853 tablet_bootstrap.cc:492] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: No bootstrap required, opened a new log
I20260619 18:47:12.965994 14853 ts_tablet_manager.cc:1397] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 18:47:12.966068 14849 ts_tablet_manager.cc:1428] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 18:47:12.966449 14546 heartbeater.cc:503] Master 127.12.234.254:40769 was elected leader, sending a full tablet report...
I20260619 18:47:12.967502 14850 raft_consensus.cc:348] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } }
I20260619 18:47:12.967989 14850 raft_consensus.cc:374] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:12.968055 14850 raft_consensus.cc:729] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d18e5b23a542548383da12f1f8c8c2, State: Initialized, Role: FOLLOWER
I20260619 18:47:12.968483 14850 consensus_queue.cc:260] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } }
I20260619 18:47:12.969169 14686 heartbeater.cc:503] Master 127.12.234.254:40769 was elected leader, sending a full tablet report...
I20260619 18:47:12.969426 14850 ts_tablet_manager.cc:1428] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260619 18:47:12.970849 14853 raft_consensus.cc:348] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } }
I20260619 18:47:12.971124 14853 raft_consensus.cc:374] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:12.971190 14853 raft_consensus.cc:729] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d542966cde50417fb3fa9bfa5e78e732, State: Initialized, Role: FOLLOWER
I20260619 18:47:12.971588 14853 consensus_queue.cc:260] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } }
I20260619 18:47:12.972292 14826 heartbeater.cc:503] Master 127.12.234.254:40769 was elected leader, sending a full tablet report...
I20260619 18:47:12.972579 14853 ts_tablet_manager.cc:1428] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260619 18:47:13.133466 14827 tablet.cc:2369] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:47:13.153734 14687 tablet.cc:2369] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:47:13.173625 14547 tablet.cc:2369] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:47:13.187470 14856 raft_consensus.cc:482] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:13.187721 14856 raft_consensus.cc:504] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } }
I20260619 18:47:13.188879 14856 leader_election.cc:288] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cacb395492042528e0c14e0e5b3e593 (127.12.234.193:34489), d542966cde50417fb3fa9bfa5e78e732 (127.12.234.195:44991)
I20260619 18:47:13.192808 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.189303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:40125 (local address 127.12.234.195:44991)
0619 18:47:13.189483 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.189487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.189673 (+ 186us) server_negotiation.cc:407] Connection header received
0619 18:47:13.189790 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.189793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.189833 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.189898 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:13.190154 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.190741 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.191885 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.192279 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.192385 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.192440 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.192616 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.192683 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260619 18:47:13.192813 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.189284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:36603 (local address 127.12.234.193:34489)
0619 18:47:13.189411 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.189416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.189595 (+ 179us) server_negotiation.cc:407] Connection header received
0619 18:47:13.189789 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.189793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.189835 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.189898 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:13.190173 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.190745 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.191881 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.192289 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.192388 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.192439 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.192631 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.192697 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:47:13.192849 14858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.189116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34489 (local address 127.12.234.194:36603)
0619 18:47:13.189492 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.189508 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.189638 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.189918 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.189921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.189956 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:13.190080 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.190086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.190846 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.190849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.191770 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.191779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.192391 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.192394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.192520 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.192541 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.192608 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.192683 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":62,"spinlock_wait_cycles":896,"thread_start_us":89,"threads_started":1}
I20260619 18:47:13.192849 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.189116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:44991 (local address 127.12.234.194:40125)
0619 18:47:13.189575 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.189588 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.189704 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.189914 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.189917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.189950 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:13.190047 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.190052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.190844 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.190847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.191776 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.191784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.192385 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.192389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.192509 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.192530 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.192584 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.192674 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":202,"spinlock_wait_cycles":16384,"thread_start_us":101,"threads_started":1}
I20260619 18:47:13.193486 14500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "569c99b5a67a4863ba7ac5cf623709cb" candidate_uuid: "f4d18e5b23a542548383da12f1f8c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cacb395492042528e0c14e0e5b3e593" is_pre_election: true
I20260619 18:47:13.193593 14780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "569c99b5a67a4863ba7ac5cf623709cb" candidate_uuid: "f4d18e5b23a542548383da12f1f8c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d542966cde50417fb3fa9bfa5e78e732" is_pre_election: true
I20260619 18:47:13.193786 14500 raft_consensus.cc:2454] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4d18e5b23a542548383da12f1f8c8c2 in term 0.
I20260619 18:47:13.193886 14780 raft_consensus.cc:2454] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4d18e5b23a542548383da12f1f8c8c2 in term 0.
I20260619 18:47:13.194262 14572 leader_election.cc:302] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cacb395492042528e0c14e0e5b3e593, f4d18e5b23a542548383da12f1f8c8c2; no voters:
I20260619 18:47:13.194546 14856 raft_consensus.cc:2781] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:13.194623 14856 raft_consensus.cc:482] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:13.194674 14856 raft_consensus.cc:3037] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:13.195668 14856 raft_consensus.cc:504] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } }
I20260619 18:47:13.195998 14856 leader_election.cc:288] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [CANDIDATE]: Term 1 election: Requested vote from peers 6cacb395492042528e0c14e0e5b3e593 (127.12.234.193:34489), d542966cde50417fb3fa9bfa5e78e732 (127.12.234.195:44991)
I20260619 18:47:13.196341 14500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "569c99b5a67a4863ba7ac5cf623709cb" candidate_uuid: "f4d18e5b23a542548383da12f1f8c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cacb395492042528e0c14e0e5b3e593"
I20260619 18:47:13.196362 14780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "569c99b5a67a4863ba7ac5cf623709cb" candidate_uuid: "f4d18e5b23a542548383da12f1f8c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d542966cde50417fb3fa9bfa5e78e732"
I20260619 18:47:13.196457 14500 raft_consensus.cc:3037] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:13.196463 14780 raft_consensus.cc:3037] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:13.197485 14500 raft_consensus.cc:2454] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4d18e5b23a542548383da12f1f8c8c2 in term 1.
I20260619 18:47:13.197788 14572 leader_election.cc:302] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cacb395492042528e0c14e0e5b3e593, f4d18e5b23a542548383da12f1f8c8c2; no voters:
I20260619 18:47:13.197893 14780 raft_consensus.cc:2454] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4d18e5b23a542548383da12f1f8c8c2 in term 1.
I20260619 18:47:13.197955 14856 raft_consensus.cc:2781] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:13.198127 14856 raft_consensus.cc:686] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 1 LEADER]: Becoming Leader. State: Replica: f4d18e5b23a542548383da12f1f8c8c2, State: Running, Role: LEADER
I20260619 18:47:13.198536 14856 consensus_queue.cc:237] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } }
I20260619 18:47:13.201768 14342 catalog_manager.cc:5495] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4d18e5b23a542548383da12f1f8c8c2 (127.12.234.194). New cstate: current_term: 1 leader_uuid: "f4d18e5b23a542548383da12f1f8c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } health_report { overall_health: HEALTHY } } }
I20260619 18:47:13.242348 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.238754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.12.234.194:35581)
0619 18:47:13.238867 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.238871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.238894 (+ 23us) server_negotiation.cc:407] Connection header received
0619 18:47:13.239071 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.239074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.239133 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.239221 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:13.239449 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.239976 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.240903 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.241149 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.241236 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.241594 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:13.242000 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:13.242085 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.242141 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.242233 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:47:13.248106 14500 raft_consensus.cc:1261] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [term 1 FOLLOWER]: Refusing update from remote peer f4d18e5b23a542548383da12f1f8c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:13.248144 14780 raft_consensus.cc:1261] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Refusing update from remote peer f4d18e5b23a542548383da12f1f8c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:13.248839 14860 consensus_queue.cc:1035] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:13.249014 14856 consensus_queue.cc:1035] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:13.256793 14863 mvcc.cc:205] Tried to move back new op lower bound from 7298641236977061888 to 7298641236783734784. Current Snapshot: MvccSnapshot[applied={T|T < 7298641236977061888}]
I20260619 18:47:13.258225 14865 mvcc.cc:205] Tried to move back new op lower bound from 7298641236977061888 to 7298641236783734784. Current Snapshot: MvccSnapshot[applied={T|T < 7298641236977061888}]
I20260619 18:47:13.258755 14864 mvcc.cc:205] Tried to move back new op lower bound from 7298641236977061888 to 7298641236783734784. Current Snapshot: MvccSnapshot[applied={T|T < 7298641236977061888}]
I20260619 18:47:13.269050 14342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:13.269217 14342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:13.271683 14342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd: Sending DeleteTablet for 3 replicas of tablet 569c99b5a67a4863ba7ac5cf623709cb
I20260619 18:47:13.272511 14480 tablet_service.cc:1514] Processing DeleteTablet for tablet 569c99b5a67a4863ba7ac5cf623709cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:13 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:45152
I20260619 18:47:13.272588 14620 tablet_service.cc:1514] Processing DeleteTablet for tablet 569c99b5a67a4863ba7ac5cf623709cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:13 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:40248
I20260619 18:47:13.272562 14760 tablet_service.cc:1514] Processing DeleteTablet for tablet 569c99b5a67a4863ba7ac5cf623709cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:13 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:40950
I20260619 18:47:13.273066 14874 tablet_replica.cc:330] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: stopping tablet replica
I20260619 18:47:13.273296 14873 tablet_replica.cc:330] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: stopping tablet replica
I20260619 18:47:13.273440 14875 tablet_replica.cc:330] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: stopping tablet replica
I20260619 18:47:13.273698 14873 raft_consensus.cc:2229] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:13.273752 14875 raft_consensus.cc:2229] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:13.274075 14875 raft_consensus.cc:2258] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:13.274142 14342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 18:47:13.274287 14873 raft_consensus.cc:2258] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 18:47:13.274618 14342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:13.275718 14875 ts_tablet_manager.cc:1905] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:13.275894 14873 ts_tablet_manager.cc:1905] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:13.276342 14874 raft_consensus.cc:2229] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:13.276666 14874 raft_consensus.cc:2258] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:13.278362 14875 ts_tablet_manager.cc:1918] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:13.278461 14875 log.cc:1196] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/569c99b5a67a4863ba7ac5cf623709cb
I20260619 18:47:13.278774 14875 ts_tablet_manager.cc:1939] T 569c99b5a67a4863ba7ac5cf623709cb P d542966cde50417fb3fa9bfa5e78e732: Deleting consensus metadata
I20260619 18:47:13.279206 14874 ts_tablet_manager.cc:1905] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:13.279999 14620 tablet_service.cc:1467] Processing CreateTablet for tablet 23266d3d5abb472481bc7b9a5f61328c (DEFAULT_TABLE table=test-table [id=d234607ea9e14fb8aedd460711cea3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:13.280051 14480 tablet_service.cc:1467] Processing CreateTablet for tablet 23266d3d5abb472481bc7b9a5f61328c (DEFAULT_TABLE table=test-table [id=d234607ea9e14fb8aedd460711cea3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:13.280144 14760 tablet_service.cc:1467] Processing CreateTablet for tablet 23266d3d5abb472481bc7b9a5f61328c (DEFAULT_TABLE table=test-table [id=d234607ea9e14fb8aedd460711cea3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:13.280254 14620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23266d3d5abb472481bc7b9a5f61328c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:13.280364 14760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23266d3d5abb472481bc7b9a5f61328c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:13.280452 14480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23266d3d5abb472481bc7b9a5f61328c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:13.282426 14850 tablet_bootstrap.cc:492] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: Bootstrap starting.
I20260619 18:47:13.282465 14853 tablet_bootstrap.cc:492] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: Bootstrap starting.
I20260619 18:47:13.282718 14849 tablet_bootstrap.cc:492] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: Bootstrap starting.
I20260619 18:47:13.283334 14853 tablet_bootstrap.cc:654] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:13.283350 14850 tablet_bootstrap.cc:654] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:13.283535 14849 tablet_bootstrap.cc:654] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:13.283897 14329 catalog_manager.cc:4841] TS d542966cde50417fb3fa9bfa5e78e732 (127.12.234.195:44991): tablet 569c99b5a67a4863ba7ac5cf623709cb (table test-table [id=94a825d8b5b544dc96bd5731da360f5b]) successfully deleted
I20260619 18:47:13.284618 14873 ts_tablet_manager.cc:1918] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:13.284761 14873 log.cc:1196] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/569c99b5a67a4863ba7ac5cf623709cb
I20260619 18:47:13.285142 14873 ts_tablet_manager.cc:1939] T 569c99b5a67a4863ba7ac5cf623709cb P f4d18e5b23a542548383da12f1f8c8c2: Deleting consensus metadata
I20260619 18:47:13.285456 14853 tablet_bootstrap.cc:492] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: No bootstrap required, opened a new log
I20260619 18:47:13.285539 14853 ts_tablet_manager.cc:1397] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:13.285952 14853 raft_consensus.cc:348] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.286063 14853 raft_consensus.cc:374] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:13.286090 14853 raft_consensus.cc:729] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d542966cde50417fb3fa9bfa5e78e732, State: Initialized, Role: FOLLOWER
I20260619 18:47:13.286235 14853 consensus_queue.cc:260] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.286291 14874 ts_tablet_manager.cc:1918] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:13.286376 14874 log.cc:1196] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/569c99b5a67a4863ba7ac5cf623709cb
I20260619 18:47:13.286491 14853 ts_tablet_manager.cc:1428] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:13.286796 14329 catalog_manager.cc:4841] TS f4d18e5b23a542548383da12f1f8c8c2 (127.12.234.194:35581): tablet 569c99b5a67a4863ba7ac5cf623709cb (table test-table [id=94a825d8b5b544dc96bd5731da360f5b]) successfully deleted
I20260619 18:47:13.287150 14874 ts_tablet_manager.cc:1939] T 569c99b5a67a4863ba7ac5cf623709cb P 6cacb395492042528e0c14e0e5b3e593: Deleting consensus metadata
I20260619 18:47:13.287714 14850 tablet_bootstrap.cc:492] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: No bootstrap required, opened a new log
I20260619 18:47:13.287804 14850 ts_tablet_manager.cc:1397] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260619 18:47:13.288239 14850 raft_consensus.cc:348] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.288345 14850 raft_consensus.cc:374] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:13.288378 14850 raft_consensus.cc:729] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d18e5b23a542548383da12f1f8c8c2, State: Initialized, Role: FOLLOWER
I20260619 18:47:13.288354 14328 catalog_manager.cc:4841] TS 6cacb395492042528e0c14e0e5b3e593 (127.12.234.193:34489): tablet 569c99b5a67a4863ba7ac5cf623709cb (table test-table [id=94a825d8b5b544dc96bd5731da360f5b]) successfully deleted
I20260619 18:47:13.288489 14850 consensus_queue.cc:260] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.288735 14850 ts_tablet_manager.cc:1428] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 18:47:13.289614 14849 tablet_bootstrap.cc:492] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: No bootstrap required, opened a new log
I20260619 18:47:13.289707 14849 ts_tablet_manager.cc:1397] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260619 18:47:13.290133 14849 raft_consensus.cc:348] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.290261 14849 raft_consensus.cc:374] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:13.290287 14849 raft_consensus.cc:729] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cacb395492042528e0c14e0e5b3e593, State: Initialized, Role: FOLLOWER
I20260619 18:47:13.290395 14849 consensus_queue.cc:260] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.290627 14849 ts_tablet_manager.cc:1428] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:13.309828 14855 raft_consensus.cc:482] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:13.309979 14855 raft_consensus.cc:504] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.311110 14855 leader_election.cc:288] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4d18e5b23a542548383da12f1f8c8c2 (127.12.234.194:35581), d542966cde50417fb3fa9bfa5e78e732 (127.12.234.195:44991)
I20260619 18:47:13.315735 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.311413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:50415 (local address 127.12.234.194:35581)
0619 18:47:13.311535 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.311539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.311697 (+ 158us) server_negotiation.cc:407] Connection header received
0619 18:47:13.311902 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.311905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.311955 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.312003 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:13.312422 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.313063 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.314440 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.315141 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.315258 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.315311 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.315517 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.315596 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 18:47:13.315863 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.311818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:48291 (local address 127.12.234.195:44991)
0619 18:47:13.311944 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.311948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.312210 (+ 262us) server_negotiation.cc:407] Connection header received
0619 18:47:13.312369 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.312372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.312421 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.312499 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:13.312828 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.313577 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.314824 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.315346 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.315465 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.315512 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.315690 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.315761 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 18:47:13.315901 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.311707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:44991 (local address 127.12.234.193:48291)
0619 18:47:13.312113 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.312126 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.312226 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.312558 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.312561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.312601 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:13.312723 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.312729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.313690 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.313694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.314701 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.314709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.315463 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.315467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.315568 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.315590 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.315662 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.315730 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":1,"spinlock_wait_cycles":25344,"thread_start_us":89,"threads_started":1}
I20260619 18:47:13.316426 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.311335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:35581 (local address 127.12.234.193:50415)
0619 18:47:13.311616 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.311630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.311777 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.312163 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.312166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.312184 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:13.312333 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.312340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.313162 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.313165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.314325 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.314335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.315269 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.315273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.315372 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.315399 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.316210 (+ 811us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.316285 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":25,"spinlock_wait_cycles":9088,"thread_start_us":90,"threads_started":1}
I20260619 18:47:13.316565 14780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23266d3d5abb472481bc7b9a5f61328c" candidate_uuid: "6cacb395492042528e0c14e0e5b3e593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d542966cde50417fb3fa9bfa5e78e732" is_pre_election: true
I20260619 18:47:13.316712 14780 raft_consensus.cc:2454] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cacb395492042528e0c14e0e5b3e593 in term 0.
I20260619 18:47:13.317016 14640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23266d3d5abb472481bc7b9a5f61328c" candidate_uuid: "6cacb395492042528e0c14e0e5b3e593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d18e5b23a542548383da12f1f8c8c2" is_pre_election: true
I20260619 18:47:13.317245 14640 raft_consensus.cc:2454] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cacb395492042528e0c14e0e5b3e593 in term 0.
I20260619 18:47:13.317654 14433 leader_election.cc:302] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cacb395492042528e0c14e0e5b3e593, d542966cde50417fb3fa9bfa5e78e732; no voters:
I20260619 18:47:13.317934 14855 raft_consensus.cc:2781] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:13.317994 14855 raft_consensus.cc:482] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:13.318027 14855 raft_consensus.cc:3037] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:13.319090 14855 raft_consensus.cc:504] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.319383 14855 leader_election.cc:288] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [CANDIDATE]: Term 1 election: Requested vote from peers f4d18e5b23a542548383da12f1f8c8c2 (127.12.234.194:35581), d542966cde50417fb3fa9bfa5e78e732 (127.12.234.195:44991)
I20260619 18:47:13.319770 14640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23266d3d5abb472481bc7b9a5f61328c" candidate_uuid: "6cacb395492042528e0c14e0e5b3e593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d18e5b23a542548383da12f1f8c8c2"
I20260619 18:47:13.319792 14780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23266d3d5abb472481bc7b9a5f61328c" candidate_uuid: "6cacb395492042528e0c14e0e5b3e593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d542966cde50417fb3fa9bfa5e78e732"
I20260619 18:47:13.319890 14640 raft_consensus.cc:3037] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:13.319895 14780 raft_consensus.cc:3037] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:13.320757 14780 raft_consensus.cc:2454] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cacb395492042528e0c14e0e5b3e593 in term 1.
I20260619 18:47:13.321012 14433 leader_election.cc:302] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cacb395492042528e0c14e0e5b3e593, d542966cde50417fb3fa9bfa5e78e732; no voters:
I20260619 18:47:13.321035 14640 raft_consensus.cc:2454] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cacb395492042528e0c14e0e5b3e593 in term 1.
I20260619 18:47:13.321187 14855 raft_consensus.cc:2781] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:13.321393 14855 raft_consensus.cc:686] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 1 LEADER]: Becoming Leader. State: Replica: 6cacb395492042528e0c14e0e5b3e593, State: Running, Role: LEADER
I20260619 18:47:13.321638 14855 consensus_queue.cc:237] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.323850 14338 catalog_manager.cc:5495] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cacb395492042528e0c14e0e5b3e593 (127.12.234.193). New cstate: current_term: 1 leader_uuid: "6cacb395492042528e0c14e0e5b3e593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } health_report { overall_health: UNKNOWN } } }
Jun 19 18:47:13 dist-test-slave-fljv krb5kdc[14298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:13.369208 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.359910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40769 (local address 127.0.0.1:39378)
0619 18:47:13.360199 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.360215 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.360324 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.360611 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.360614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.361041 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:13.361151 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.361158 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.362244 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.362247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.362807 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.362814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.363363 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.363365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.363430 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.363977 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:13.363989 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:13.365674 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:13.367669 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:13.367677 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:13.367682 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:13.367913 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:13.368181 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:13.368183 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:13.368184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:13.368258 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:13.368676 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:13.368690 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:13.368796 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.368885 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.369006 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
I20260619 18:47:13.369251 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.359973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39378 (local address 127.12.234.254:40769)
0619 18:47:13.360105 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.360110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.360289 (+ 179us) server_negotiation.cc:407] Connection header received
0619 18:47:13.360404 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.360407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.360457 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.360551 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:13.361286 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.362068 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.362916 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.363240 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.363338 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.365818 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:13.365823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:13.365825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:13.365851 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:13.367546 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:13.368000 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:13.368004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:13.368006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:13.368093 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:13.368332 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:13.368335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:13.368336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:13.368573 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:13.368671 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.368951 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.369067 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 18:47:13.373317 14338 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:39378:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 18:47:13.374063 14338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:13.379170 14480 tablet_service.cc:1467] Processing CreateTablet for tablet c865694f306149748afd0c992bb821f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80ddeea71ad84b728fc04859f13f8aeb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 18:47:13.379221 14620 tablet_service.cc:1467] Processing CreateTablet for tablet c865694f306149748afd0c992bb821f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80ddeea71ad84b728fc04859f13f8aeb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 18:47:13.379348 14760 tablet_service.cc:1467] Processing CreateTablet for tablet c865694f306149748afd0c992bb821f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80ddeea71ad84b728fc04859f13f8aeb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 18:47:13.379583 14480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c865694f306149748afd0c992bb821f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:13.379583 14620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c865694f306149748afd0c992bb821f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:13.379705 14760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c865694f306149748afd0c992bb821f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:13.382432 14850 tablet_bootstrap.cc:492] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: Bootstrap starting.
I20260619 18:47:13.382835 14853 tablet_bootstrap.cc:492] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: Bootstrap starting.
I20260619 18:47:13.383301 14850 tablet_bootstrap.cc:654] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:13.383806 14853 tablet_bootstrap.cc:654] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:13.385023 14850 tablet_bootstrap.cc:492] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: No bootstrap required, opened a new log
I20260619 18:47:13.385145 14850 ts_tablet_manager.cc:1397] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260619 18:47:13.385720 14850 raft_consensus.cc:348] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.385815 14850 raft_consensus.cc:374] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:13.385834 14850 raft_consensus.cc:729] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d18e5b23a542548383da12f1f8c8c2, State: Initialized, Role: FOLLOWER
I20260619 18:47:13.385835 14853 tablet_bootstrap.cc:492] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: No bootstrap required, opened a new log
I20260619 18:47:13.385921 14853 ts_tablet_manager.cc:1397] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:13.385938 14850 consensus_queue.cc:260] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.386229 14850 ts_tablet_manager.cc:1428] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:13.386431 14868 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.386564 14868 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:13.386412 14853 raft_consensus.cc:348] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.386915 14853 raft_consensus.cc:374] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:13.386965 14853 raft_consensus.cc:729] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d542966cde50417fb3fa9bfa5e78e732, State: Initialized, Role: FOLLOWER
I20260619 18:47:13.387203 14849 tablet_bootstrap.cc:492] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: Bootstrap starting.
I20260619 18:47:13.388172 14849 tablet_bootstrap.cc:654] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:13.388044 14853 consensus_queue.cc:260] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.388363 14857 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.388515 14857 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:13.389637 14849 tablet_bootstrap.cc:492] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: No bootstrap required, opened a new log
I20260619 18:47:13.389717 14849 ts_tablet_manager.cc:1397] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:13.390122 14849 raft_consensus.cc:348] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.390323 14849 raft_consensus.cc:374] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:13.390365 14849 raft_consensus.cc:729] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cacb395492042528e0c14e0e5b3e593, State: Initialized, Role: FOLLOWER
I20260619 18:47:13.390450 14853 ts_tablet_manager.cc:1428] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260619 18:47:13.390489 14849 consensus_queue.cc:260] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.390741 14849 ts_tablet_manager.cc:1428] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:13.390925 14855 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.391059 14855 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:13.548751 14857 raft_consensus.cc:482] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:13.548918 14857 raft_consensus.cc:504] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.550077 14857 leader_election.cc:288] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cacb395492042528e0c14e0e5b3e593 (127.12.234.193:34489), f4d18e5b23a542548383da12f1f8c8c2 (127.12.234.194:35581)
I20260619 18:47:13.553961 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.550463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:57079 (local address 127.12.234.194:35581)
0619 18:47:13.550592 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.550596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.550780 (+ 184us) server_negotiation.cc:407] Connection header received
0619 18:47:13.550915 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.550917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.550957 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.551027 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:13.551327 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.551834 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.552994 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.553377 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.553480 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.553535 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.553753 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.553826 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 18:47:13.553980 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.550421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:39759 (local address 127.12.234.193:34489)
0619 18:47:13.550571 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.550577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.550706 (+ 129us) server_negotiation.cc:407] Connection header received
0619 18:47:13.550868 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.550872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.550933 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.551009 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:13.551262 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.551800 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.552975 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.553378 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.553506 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.553564 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.553763 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.553842 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 18:47:13.554028 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.550286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34489 (local address 127.12.234.195:39759)
0619 18:47:13.550604 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.550617 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.550737 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.551025 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.551027 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.551042 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:13.551154 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.551161 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.551908 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.551912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.552852 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.552861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.553492 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.553495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.553644 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.553663 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.553730 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.553828 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":113,"thread_start_us":86,"threads_started":1}
I20260619 18:47:13.554047 14893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.550363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:35581 (local address 127.12.234.195:57079)
0619 18:47:13.550702 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.550714 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.550815 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.551084 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.551087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.551105 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:13.551208 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.551222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.551954 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.551957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.552869 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.552878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.553494 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.553499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.553631 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.553654 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.553726 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.553855 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":68,"thread_start_us":104,"threads_started":1}
I20260619 18:47:13.554477 14500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c865694f306149748afd0c992bb821f6" candidate_uuid: "d542966cde50417fb3fa9bfa5e78e732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cacb395492042528e0c14e0e5b3e593" is_pre_election: true
I20260619 18:47:13.554538 14640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c865694f306149748afd0c992bb821f6" candidate_uuid: "d542966cde50417fb3fa9bfa5e78e732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d18e5b23a542548383da12f1f8c8c2" is_pre_election: true
I20260619 18:47:13.554637 14500 raft_consensus.cc:2454] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d542966cde50417fb3fa9bfa5e78e732 in term 0.
I20260619 18:47:13.554682 14640 raft_consensus.cc:2454] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d542966cde50417fb3fa9bfa5e78e732 in term 0.
I20260619 18:47:13.555071 14713 leader_election.cc:302] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d542966cde50417fb3fa9bfa5e78e732, f4d18e5b23a542548383da12f1f8c8c2; no voters:
I20260619 18:47:13.555367 14857 raft_consensus.cc:2781] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:13.555420 14857 raft_consensus.cc:482] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:13.555452 14857 raft_consensus.cc:3037] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:13.556380 14857 raft_consensus.cc:504] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.556721 14857 leader_election.cc:288] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [CANDIDATE]: Term 1 election: Requested vote from peers 6cacb395492042528e0c14e0e5b3e593 (127.12.234.193:34489), f4d18e5b23a542548383da12f1f8c8c2 (127.12.234.194:35581)
I20260619 18:47:13.557039 14500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c865694f306149748afd0c992bb821f6" candidate_uuid: "d542966cde50417fb3fa9bfa5e78e732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cacb395492042528e0c14e0e5b3e593"
I20260619 18:47:13.557163 14500 raft_consensus.cc:3037] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:13.557132 14640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c865694f306149748afd0c992bb821f6" candidate_uuid: "d542966cde50417fb3fa9bfa5e78e732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d18e5b23a542548383da12f1f8c8c2"
I20260619 18:47:13.557251 14640 raft_consensus.cc:3037] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:13.558009 14500 raft_consensus.cc:2454] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d542966cde50417fb3fa9bfa5e78e732 in term 1.
I20260619 18:47:13.558236 14640 raft_consensus.cc:2454] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d542966cde50417fb3fa9bfa5e78e732 in term 1.
I20260619 18:47:13.558326 14712 leader_election.cc:302] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cacb395492042528e0c14e0e5b3e593, d542966cde50417fb3fa9bfa5e78e732; no voters:
I20260619 18:47:13.558508 14857 raft_consensus.cc:2781] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:13.558704 14857 raft_consensus.cc:686] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [term 1 LEADER]: Becoming Leader. State: Replica: d542966cde50417fb3fa9bfa5e78e732, State: Running, Role: LEADER
I20260619 18:47:13.558964 14857 consensus_queue.cc:237] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } }
I20260619 18:47:13.560276 14894 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: TxnStatusTablet state changed. Reason: New leader d542966cde50417fb3fa9bfa5e78e732. Latest consensus state: current_term: 1 leader_uuid: "d542966cde50417fb3fa9bfa5e78e732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.560382 14894 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: This TxnStatusTablet replica's current role is: LEADER
I20260619 18:47:13.561008 14896 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 18:47:13.561161 14896 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 18:47:13.561506 14338 catalog_manager.cc:5495] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 reported cstate change: term changed from 0 to 1, leader changed from <none> to d542966cde50417fb3fa9bfa5e78e732 (127.12.234.195). New cstate: current_term: 1 leader_uuid: "d542966cde50417fb3fa9bfa5e78e732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } health_report { overall_health: HEALTHY } } }
I20260619 18:47:13.588371 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.583822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:44991 (local address 127.0.0.1:40960)
0619 18:47:13.584001 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.584019 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.584119 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.584454 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.584456 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.584462 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:13.584579 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.584584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.585575 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.585578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.586267 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.586277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.586960 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.586963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.587110 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.587231 (+ 121us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:13.588049 (+ 818us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:13.588065 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.588141 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.588231 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 18:47:13.588371 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.583961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.12.234.195:44991)
0619 18:47:13.584093 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.584097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.584112 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:13.584258 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.584261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.584334 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.584438 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:13.584675 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.585447 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.586374 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.586761 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.586932 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.587380 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:13.587922 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:13.588031 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.588174 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.588231 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 18:47:13.591460 14500 raft_consensus.cc:1261] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593 [term 1 FOLLOWER]: Refusing update from remote peer d542966cde50417fb3fa9bfa5e78e732: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:13.591634 14640 raft_consensus.cc:1261] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2 [term 1 FOLLOWER]: Refusing update from remote peer d542966cde50417fb3fa9bfa5e78e732: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:13.592073 14894 consensus_queue.cc:1035] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:13.592242 14857 consensus_queue.cc:1035] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:13.593897 14868 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: TxnStatusTablet state changed. Reason: New leader d542966cde50417fb3fa9bfa5e78e732. Latest consensus state: current_term: 1 leader_uuid: "d542966cde50417fb3fa9bfa5e78e732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.594002 14868 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:13.595157 14855 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: TxnStatusTablet state changed. Reason: New leader d542966cde50417fb3fa9bfa5e78e732. Latest consensus state: current_term: 1 leader_uuid: "d542966cde50417fb3fa9bfa5e78e732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.595274 14855 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:13.596400 14894 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d542966cde50417fb3fa9bfa5e78e732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.596503 14894 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: This TxnStatusTablet replica's current role is: LEADER
I20260619 18:47:13.596612 14894 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d542966cde50417fb3fa9bfa5e78e732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.596685 14894 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P d542966cde50417fb3fa9bfa5e78e732: This TxnStatusTablet replica's current role is: LEADER
I20260619 18:47:13.596774 14855 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d542966cde50417fb3fa9bfa5e78e732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.596866 14855 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P 6cacb395492042528e0c14e0e5b3e593: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:13.599311 14868 tablet_replica.cc:439] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d542966cde50417fb3fa9bfa5e78e732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cacb395492042528e0c14e0e5b3e593" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34489 } } peers { permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 } } peers { permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 } } }
I20260619 18:47:13.599416 14868 tablet_replica.cc:442] T c865694f306149748afd0c992bb821f6 P f4d18e5b23a542548383da12f1f8c8c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:13.605460 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.601667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.12.234.193:34489)
0619 18:47:13.601770 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.601774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.601789 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:13.601913 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.601916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.601973 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.602081 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:13.602289 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.603057 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.603853 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.604202 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.604295 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.604519 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:13.605049 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:13.605164 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.605265 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.605325 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:47:13.616575 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.612705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:38991 (local address 127.12.234.195:44991)
0619 18:47:13.612807 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.612810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.613066 (+ 256us) server_negotiation.cc:407] Connection header received
0619 18:47:13.613184 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.613187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.613240 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.613313 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:13.613588 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.614326 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.615588 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.615812 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.615897 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.616203 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:13.616272 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:13.616324 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.616420 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.616462 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 18:47:13.616653 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.612579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:44991 (local address 127.12.234.193:38991)
0619 18:47:13.612981 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.612995 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.613094 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.613335 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.613338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.613346 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:13.613472 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.613478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.614432 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.614435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.615461 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.615472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.615918 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.615921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.616016 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.616069 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:13.616346 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:13.616352 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.616411 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.616487 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":113,"threads_started":1}
I20260619 18:47:13.624188 14640 raft_consensus.cc:1261] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [term 1 FOLLOWER]: Refusing update from remote peer 6cacb395492042528e0c14e0e5b3e593: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:13.624296 14780 raft_consensus.cc:1261] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Refusing update from remote peer 6cacb395492042528e0c14e0e5b3e593: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:13.624732 14881 consensus_queue.cc:1035] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4d18e5b23a542548383da12f1f8c8c2" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:13.624918 14855 consensus_queue.cc:1035] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d542966cde50417fb3fa9bfa5e78e732" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 44991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:13.648293 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.644452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:37645 (local address 127.12.234.193:34489)
0619 18:47:13.644560 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.644566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.644580 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:13.644714 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.644718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.644767 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.644811 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:13.645218 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.645765 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.647075 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.647350 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.647460 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.647801 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:13.647898 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:13.647974 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.648084 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.648150 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:47:13.648723 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.644132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34489 (local address 127.12.234.195:37645)
0619 18:47:13.644482 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:13.644497 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:13.644591 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:13.644940 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:13.644943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:13.644961 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:13.645071 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.645076 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.645871 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.645874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.646964 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:13.646976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.647473 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.647477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:13.647575 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.647634 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:13.647981 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:13.647987 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:47:13.648491 (+ 504us) client_negotiation.cc:240] Negotiation successful
0619 18:47:13.648574 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":134,"threads_started":1}
I20260619 18:47:13.678938 14342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:13.679121 14342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:13.681402 14342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e179312d10164a3d841c2c99734624dd: Sending DeleteTablet for 3 replicas of tablet 23266d3d5abb472481bc7b9a5f61328c
I20260619 18:47:13.682149 14620 tablet_service.cc:1514] Processing DeleteTablet for tablet 23266d3d5abb472481bc7b9a5f61328c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:13 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:40248
I20260619 18:47:13.682361 14873 tablet_replica.cc:330] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: stopping tablet replica
I20260619 18:47:13.682341 14480 tablet_service.cc:1514] Processing DeleteTablet for tablet 23266d3d5abb472481bc7b9a5f61328c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:13 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:45152
I20260619 18:47:13.682475 14873 raft_consensus.cc:2229] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:13.682503 14874 tablet_replica.cc:330] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: stopping tablet replica
I20260619 18:47:13.682591 14874 raft_consensus.cc:2229] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:13.682602 14873 raft_consensus.cc:2258] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:13.682873 14874 raft_consensus.cc:2258] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:13.682837 14760 tablet_service.cc:1514] Processing DeleteTablet for tablet 23266d3d5abb472481bc7b9a5f61328c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:13 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:40950
I20260619 18:47:13.683017 14875 tablet_replica.cc:330] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: stopping tablet replica
I20260619 18:47:13.683133 14875 raft_consensus.cc:2229] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:13.683239 14875 raft_consensus.cc:2258] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:13.683570 14873 ts_tablet_manager.cc:1905] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:13.684903 14875 ts_tablet_manager.cc:1905] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:13.686472 14873 ts_tablet_manager.cc:1918] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:13.686540 14873 log.cc:1196] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/23266d3d5abb472481bc7b9a5f61328c
I20260619 18:47:13.686793 14873 ts_tablet_manager.cc:1939] T 23266d3d5abb472481bc7b9a5f61328c P f4d18e5b23a542548383da12f1f8c8c2: Deleting consensus metadata
I20260619 18:47:13.686769 14874 ts_tablet_manager.cc:1905] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:13.687458 14329 catalog_manager.cc:4841] TS f4d18e5b23a542548383da12f1f8c8c2 (127.12.234.194:35581): tablet 23266d3d5abb472481bc7b9a5f61328c (table test-table [id=d234607ea9e14fb8aedd460711cea3ac]) successfully deleted
I20260619 18:47:13.690725 14874 ts_tablet_manager.cc:1918] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:13.690802 14874 log.cc:1196] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/23266d3d5abb472481bc7b9a5f61328c
I20260619 18:47:13.691035 14874 ts_tablet_manager.cc:1939] T 23266d3d5abb472481bc7b9a5f61328c P 6cacb395492042528e0c14e0e5b3e593: Deleting consensus metadata
I20260619 18:47:13.691018 14875 ts_tablet_manager.cc:1918] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:13.691088 14875 log.cc:1196] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/23266d3d5abb472481bc7b9a5f61328c
I20260619 18:47:13.691322 14875 ts_tablet_manager.cc:1939] T 23266d3d5abb472481bc7b9a5f61328c P d542966cde50417fb3fa9bfa5e78e732: Deleting consensus metadata
Jun 19 18:47:13 dist-test-slave-fljv krb5kdc[14298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:13.691751 14328 catalog_manager.cc:4841] TS 6cacb395492042528e0c14e0e5b3e593 (127.12.234.193:34489): tablet 23266d3d5abb472481bc7b9a5f61328c (table test-table [id=d234607ea9e14fb8aedd460711cea3ac]) successfully deleted
I20260619 18:47:13.691956 14329 catalog_manager.cc:4841] TS d542966cde50417fb3fa9bfa5e78e732 (127.12.234.195:44991): tablet 23266d3d5abb472481bc7b9a5f61328c (table test-table [id=d234607ea9e14fb8aedd460711cea3ac]) successfully deleted
I20260619 18:47:13.695679 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.684420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.12.234.193:34489)
0619 18:47:13.684521 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.684525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.685049 (+ 524us) server_negotiation.cc:407] Connection header received
0619 18:47:13.685304 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.685308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.685354 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.685411 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:13.686377 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.687109 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.688229 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.688559 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.688663 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.692282 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:13.692292 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:13.692307 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:13.692331 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:13.693926 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:13.694402 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:13.694408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:13.694409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:13.694492 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:13.694745 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:13.694748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:13.694750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:13.694987 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:13.695182 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.695388 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.695499 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 18:47:13.696096 14531 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45170
I20260619 18:47:13.704533 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:13.697634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39382 (local address 127.12.234.254:40769)
0619 18:47:13.697758 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:13.697762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:13.698165 (+ 403us) server_negotiation.cc:407] Connection header received
0619 18:47:13.698300 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:13.698303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:13.698353 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:13.698411 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:13.699000 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.699510 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.700273 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:13.700496 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:13.700588 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:13.701755 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:13.701759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:13.701762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:13.701788 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:13.703097 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:13.703457 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:13.703461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:13.703462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:13.703535 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:13.703770 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:13.703773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:13.703775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:13.703941 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:13.704051 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:13.704280 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 18:47:13.704387 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 18:47:13.705051 14342 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39382
I20260619 18:47:13.706421 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 14415
I20260619 18:47:13.712994 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 14555
I20260619 18:47:13.719035 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 14695
I20260619 18:47:13.724769 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 14311
2026-06-19T18:47:13Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2884 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:13 dist-test-slave-fljv krb5kdc[14919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:13 dist-test-slave-fljv krb5kdc[14919](info): set up 2 sockets
Jun 19 18:47:13 dist-test-slave-fljv krb5kdc[14919](info): commencing operation
krb5kdc: starting...
W20260619 18:47:14.681828 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:14 dist-test-slave-fljv krb5kdc[14919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:14.802897 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:43431
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:43431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:14.912350 14932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:14.912650 14932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:14.912693 14932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:14.916704 14932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:14.916772 14932 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 18:47:14.916788 14932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:14.916806 14932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:14.916821 14932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:14.916833 14932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:14.922276 14932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:43431
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:43431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:14.923367 14932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:14.924419 14932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:14.932616 14937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:14.932627 14940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:14.932627 14938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:14.932941 14932 server_base.cc:1034] running on GCE node
I20260619 18:47:14.933501 14932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:14.934716 14932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:14.935865 14932 hybrid_clock.cc:648] HybridClock initialized: now 1781894834935841 us; error 37 us; skew 500 ppm
Jun 19 18:47:14 dist-test-slave-fljv krb5kdc[14919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894834, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:14.938629 14932 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:14.939601 14932 webserver.cc:460] Webserver started at http://127.12.234.254:33881/ using document root <none> and password file <none>
I20260619 18:47:14.940155 14932 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:14.940227 14932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:14.940436 14932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:14.942592 14932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "128c380803c446148152adaaa32c2bfc"
format_stamp: "Formatted at 2026-06-19 18:47:14 on dist-test-slave-fljv"
server_key: "e782544e5f1316553868e8aabc14b1af"
server_key_iv: "da2ffd393596daa984f206801edf75d0"
server_key_version: "encryptionkey@0"
I20260619 18:47:14.943174 14932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "128c380803c446148152adaaa32c2bfc"
format_stamp: "Formatted at 2026-06-19 18:47:14 on dist-test-slave-fljv"
server_key: "e782544e5f1316553868e8aabc14b1af"
server_key_iv: "da2ffd393596daa984f206801edf75d0"
server_key_version: "encryptionkey@0"
I20260619 18:47:14.947497 14932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 18:47:14.949939 14947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:14.951236 14932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260619 18:47:14.951357 14932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "128c380803c446148152adaaa32c2bfc"
format_stamp: "Formatted at 2026-06-19 18:47:14 on dist-test-slave-fljv"
server_key: "e782544e5f1316553868e8aabc14b1af"
server_key_iv: "da2ffd393596daa984f206801edf75d0"
server_key_version: "encryptionkey@0"
I20260619 18:47:14.951462 14932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:14.972419 14932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:14.975514 14932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:14.975718 14932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:14.983177 14932 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:43431
I20260619 18:47:14.983201 15009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:43431 every 8 connection(s)
I20260619 18:47:14.984120 14932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:14.986894 15010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:14.989199 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 14932
I20260619 18:47:14.989300 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:14.989537 13227 external_mini_cluster.cc:1428] Setting key cda87e6475393c7f1242c280963e9b85
I20260619 18:47:14.992689 15010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc: Bootstrap starting.
I20260619 18:47:14.994989 15010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:14.995707 15010 log.cc:824] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:14.997663 15010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc: No bootstrap required, opened a new log
Jun 19 18:47:14 dist-test-slave-fljv krb5kdc[14919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:15.001264 15010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128c380803c446148152adaaa32c2bfc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43431 } }
I20260619 18:47:15.001503 15010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:15.001543 15010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128c380803c446148152adaaa32c2bfc, State: Initialized, Role: FOLLOWER
I20260619 18:47:15.002002 15010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128c380803c446148152adaaa32c2bfc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43431 } }
I20260619 18:47:15.002143 15010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:15.002226 15010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:15.002314 15010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:15.003187 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:14.991013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.12.234.254:43431)
0619 18:47:14.991448 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:14.991462 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:14.991505 (+ 43us) server_negotiation.cc:407] Connection header received
0619 18:47:14.992133 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:14.992149 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:14.992486 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:14.992804 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:14.993595 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:14.994677 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:14.995559 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:14.996075 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:14.996221 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:14.998758 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:14.998763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:14.998776 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:14.998804 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:15.000339 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.000761 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.000766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.000771 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.000882 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.001194 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.001197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.001199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.001653 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:15.001854 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.002085 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.002428 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260619 18:47:15.003196 15010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128c380803c446148152adaaa32c2bfc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43431 } }
I20260619 18:47:15.003548 15010 leader_election.cc:302] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 128c380803c446148152adaaa32c2bfc; no voters:
I20260619 18:47:15.003829 15010 leader_election.cc:288] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:15.003948 15015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:15.004155 15015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [term 1 LEADER]: Becoming Leader. State: Replica: 128c380803c446148152adaaa32c2bfc, State: Running, Role: LEADER
I20260619 18:47:15.004441 15015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128c380803c446148152adaaa32c2bfc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43431 } }
I20260619 18:47:15.005092 15010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:15.005468 15017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "128c380803c446148152adaaa32c2bfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128c380803c446148152adaaa32c2bfc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43431 } } }
I20260619 18:47:15.005477 15015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 128c380803c446148152adaaa32c2bfc. Latest consensus state: current_term: 1 leader_uuid: "128c380803c446148152adaaa32c2bfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128c380803c446148152adaaa32c2bfc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43431 } } }
I20260619 18:47:15.005612 15017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:15.005617 15015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:15.005985 15019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:15.008998 15019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:15.018501 15019 catalog_manager.cc:1347] Generated new cluster ID: be397f0576a846d495a25fb4659f0b7f
I20260619 18:47:15.018576 15019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:15.024365 15019 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:15.025694 15019 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:15.037046 15019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc: Generated new TSK 0
I20260619 18:47:15.037904 15019 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:15.074488 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:43431
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:15.202282 15036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:15.202565 15036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:15.202632 15036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:15.206771 15036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:15.206840 15036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:15.206895 15036 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 18:47:15.206923 15036 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 18:47:15.206970 15036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:15.212347 15036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.254:43431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:15.213403 15036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:15.214444 15036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:47:15.224213 15036 server_base.cc:1034] running on GCE node
W20260619 18:47:15.224159 15041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:15.224160 15042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:15.224162 15044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:15.224871 15036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:15.225546 15036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:15.226713 15036 hybrid_clock.cc:648] HybridClock initialized: now 1781894835226685 us; error 42 us; skew 500 ppm
Jun 19 18:47:15 dist-test-slave-fljv krb5kdc[14919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894835, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:15.235450 15036 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:15.236442 15036 webserver.cc:460] Webserver started at http://127.12.234.193:41511/ using document root <none> and password file <none>
I20260619 18:47:15.237017 15036 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:15.237078 15036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:15.237274 15036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:15.239409 15036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "940d425fb5ed45f48d81977dd94ac2c9"
format_stamp: "Formatted at 2026-06-19 18:47:15 on dist-test-slave-fljv"
server_key: "2f6be395b00e6543c550b4d148ced532"
server_key_iv: "5ffd75059aa6f18f54ea1599b2ca3f66"
server_key_version: "encryptionkey@0"
I20260619 18:47:15.240002 15036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "940d425fb5ed45f48d81977dd94ac2c9"
format_stamp: "Formatted at 2026-06-19 18:47:15 on dist-test-slave-fljv"
server_key: "2f6be395b00e6543c550b4d148ced532"
server_key_iv: "5ffd75059aa6f18f54ea1599b2ca3f66"
server_key_version: "encryptionkey@0"
I20260619 18:47:15.243968 15036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:15.246632 15051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:15.247953 15036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:15.248057 15036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "940d425fb5ed45f48d81977dd94ac2c9"
format_stamp: "Formatted at 2026-06-19 18:47:15 on dist-test-slave-fljv"
server_key: "2f6be395b00e6543c550b4d148ced532"
server_key_iv: "5ffd75059aa6f18f54ea1599b2ca3f66"
server_key_version: "encryptionkey@0"
I20260619 18:47:15.248173 15036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:15.272989 15036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:15.276223 15036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:15.276397 15036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:15.277904 15036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:15.277951 15036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:15.277992 15036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:15.278019 15036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 18:47:15 dist-test-slave-fljv krb5kdc[14919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894835, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:15.288385 15036 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:39741
I20260619 18:47:15.288442 15166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:39741 every 8 connection(s)
I20260619 18:47:15.289289 15036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:15.291913 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.279089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:60297 (local address 127.12.234.254:43431)
0619 18:47:15.279205 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.279208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.280292 (+ 1084us) server_negotiation.cc:407] Connection header received
0619 18:47:15.281432 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.281435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.281491 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.281592 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:15.283394 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.284028 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.285250 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.285566 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.285668 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.288480 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:15.288484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:15.288487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:15.288519 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:15.290098 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.290577 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.290581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.290583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.290669 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.290914 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.290916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.290917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.291144 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:15.291219 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.291642 (+ 423us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.291745 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:47:15.291992 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 15036
I20260619 18:47:15.292089 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:15.292371 13227 external_mini_cluster.cc:1428] Setting key 0541c9bf9a244f69ef7a9efb62e4ff18
I20260619 18:47:15.292827 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.279604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43431 (local address 127.12.234.193:60297)
0619 18:47:15.280146 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.280191 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.281205 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.281888 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.281896 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.282439 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:15.283162 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.283184 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.284190 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.284193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.285129 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.285140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.285718 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.285722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.285883 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.286488 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:15.286509 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:15.288274 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:15.290219 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.290225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.290240 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.290485 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.290732 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.290734 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.290736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.290850 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.291219 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:15.291223 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:15.291452 (+ 229us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.291595 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.291872 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":117,"threads_started":1}
I20260619 18:47:15.294539 15167 heartbeater.cc:347] Connected to a master server at 127.12.234.254:43431
I20260619 18:47:15.294694 15167 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:15.295222 15167 heartbeater.cc:511] Master 127.12.234.254:43431 requested a full tablet report, sending...
I20260619 18:47:15.296634 14964 ts_manager.cc:194] Registered new tserver with Master: 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741)
I20260619 18:47:15.298106 14964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:60297
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:15.303898 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.296717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:48053 (local address 127.12.234.254:43431)
0619 18:47:15.296848 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.296852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.297091 (+ 239us) server_negotiation.cc:407] Connection header received
0619 18:47:15.297187 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.297190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.297232 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.297293 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:15.297705 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.298363 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.299276 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.299601 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.299674 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.300950 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:15.300954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:15.300957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:15.300981 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:15.302332 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.302766 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.302770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.302772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.302848 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.303086 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.303089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.303091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.303260 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:15.303370 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.303608 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.303713 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 18:47:15.303906 15174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.296647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43431 (local address 127.12.234.193:48053)
0619 18:47:15.297008 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.297018 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.297106 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.297316 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.297319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.297496 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:15.297608 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.297613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.298482 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.298485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.299151 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.299161 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.299816 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.299819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.299903 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.300422 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:15.300436 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:15.300826 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:15.302434 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.302437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.302440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.302673 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.302924 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.302927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.302930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.303006 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.303370 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:15.303373 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:15.303463 (+ 90us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.303551 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.303701 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":82,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:15.319262 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:43431
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:15.432665 15176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:15.432958 15176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:15.433009 15176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:15.437278 15176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:15.437350 15176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:15.437392 15176 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 18:47:15.437420 15176 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 18:47:15.437474 15176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:15.442991 15176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.254:43431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:15.444144 15176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:15.445235 15176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:15.455144 15184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:15.455147 15181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:15.455173 15182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:15.455415 15176 server_base.cc:1034] running on GCE node
I20260619 18:47:15.455952 15176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:15.456666 15176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:15.457841 15176 hybrid_clock.cc:648] HybridClock initialized: now 1781894835457821 us; error 37 us; skew 500 ppm
Jun 19 18:47:15 dist-test-slave-fljv krb5kdc[14919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894835, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:15.460628 15176 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:15.461549 15176 webserver.cc:460] Webserver started at http://127.12.234.194:35183/ using document root <none> and password file <none>
I20260619 18:47:15.462098 15176 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:15.462157 15176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:15.462373 15176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:15.464480 15176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "08edb91831b34a088ad1f11ec9d99dc4"
format_stamp: "Formatted at 2026-06-19 18:47:15 on dist-test-slave-fljv"
server_key: "3f1d57ee68ca40b8b1ce9c7855cb255b"
server_key_iv: "2572e57d9f14a2a6f3af238f8abbdf7e"
server_key_version: "encryptionkey@0"
I20260619 18:47:15.465031 15176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "08edb91831b34a088ad1f11ec9d99dc4"
format_stamp: "Formatted at 2026-06-19 18:47:15 on dist-test-slave-fljv"
server_key: "3f1d57ee68ca40b8b1ce9c7855cb255b"
server_key_iv: "2572e57d9f14a2a6f3af238f8abbdf7e"
server_key_version: "encryptionkey@0"
I20260619 18:47:15.468927 15176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:15.471637 15191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:15.473058 15176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:15.473178 15176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "08edb91831b34a088ad1f11ec9d99dc4"
format_stamp: "Formatted at 2026-06-19 18:47:15 on dist-test-slave-fljv"
server_key: "3f1d57ee68ca40b8b1ce9c7855cb255b"
server_key_iv: "2572e57d9f14a2a6f3af238f8abbdf7e"
server_key_version: "encryptionkey@0"
I20260619 18:47:15.473297 15176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:15.498222 15176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:15.501691 15176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:15.501871 15176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:15.503535 15176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:15.503590 15176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:15.503631 15176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:15.503645 15176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 18:47:15 dist-test-slave-fljv krb5kdc[14919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894835, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:15.513381 15176 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:33307
I20260619 18:47:15.513866 15306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:33307 every 8 connection(s)
I20260619 18:47:15.514323 15176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:15.516132 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 15176
I20260619 18:47:15.516240 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:15.516496 13227 external_mini_cluster.cc:1428] Setting key 15377dc442e06a929be4b6527fe10f71
I20260619 18:47:15.517424 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.504332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:51961 (local address 127.12.234.254:43431)
0619 18:47:15.504480 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.504485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.505472 (+ 987us) server_negotiation.cc:407] Connection header received
0619 18:47:15.506842 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.506847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.506910 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.506987 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:15.508493 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.509062 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.510418 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.510648 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.510750 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.513548 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:15.513553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:15.513556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:15.513589 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:15.515314 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.515785 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.515788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.515789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.515877 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.516306 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.516311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.516313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.516561 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:15.516657 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.517177 (+ 520us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.517303 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 18:47:15.518393 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.504817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43431 (local address 127.12.234.194:51961)
0619 18:47:15.505306 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.505359 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.506536 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.507364 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.507374 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.507818 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:15.508329 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.508345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.509207 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.509210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.510301 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.510312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.510802 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.510807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.510992 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.511601 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:15.511634 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:15.513352 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:15.515422 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.515427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.515438 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.515685 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.515959 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.515962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.515963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.516174 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.516662 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:15.516669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:15.516954 (+ 285us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.517117 (+ 163us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.517458 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":108,"threads_started":1}
I20260619 18:47:15.521737 15307 heartbeater.cc:347] Connected to a master server at 127.12.234.254:43431
I20260619 18:47:15.521898 15307 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:15.522535 15307 heartbeater.cc:511] Master 127.12.234.254:43431 requested a full tablet report, sending...
I20260619 18:47:15.523651 14964 ts_manager.cc:194] Registered new tserver with Master: 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194:33307)
I20260619 18:47:15.524339 14964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:51961
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:47:15.529769 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.522014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:40721 (local address 127.12.234.254:43431)
0619 18:47:15.522228 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.522233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.522500 (+ 267us) server_negotiation.cc:407] Connection header received
0619 18:47:15.522637 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.522641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.522700 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.522780 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:15.523448 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.524165 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.525353 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.525666 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.525765 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.526903 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:15.526906 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:15.526908 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:15.526931 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:15.528074 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.528568 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.528571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.528572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.528653 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.528930 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.528933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.528935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.529164 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:15.529250 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.529496 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.529614 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260619 18:47:15.529779 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.522012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43431 (local address 127.12.234.194:40721)
0619 18:47:15.522390 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.522405 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.522518 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.522829 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.522832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.523054 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:15.523181 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.523187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.524305 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.524309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.525221 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.525234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.525805 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.525810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.525904 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.526425 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:15.526437 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:15.526780 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:15.528197 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.528200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.528202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.528444 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.528760 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.528763 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.528766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.528851 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.529255 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:15.529259 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:15.529346 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.529438 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.529591 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:15.545331 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:43431
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:15.655405 15316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:15.655691 15316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:15.655741 15316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:15.659927 15316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:15.660000 15316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:15.660040 15316 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 18:47:15.660053 15316 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 18:47:15.660085 15316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:15.665495 15316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.254:43431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:15.666591 15316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:15.667613 15316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:15.677088 15321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:15.677268 15316 server_base.cc:1034] running on GCE node
W20260619 18:47:15.677018 15322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:15.677021 15324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:15.677850 15316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:15.678566 15316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:15.679735 15316 hybrid_clock.cc:648] HybridClock initialized: now 1781894835679715 us; error 26 us; skew 500 ppm
Jun 19 18:47:15 dist-test-slave-fljv krb5kdc[14919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894835, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:15.682441 15316 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:15.683350 15316 webserver.cc:460] Webserver started at http://127.12.234.195:38911/ using document root <none> and password file <none>
I20260619 18:47:15.683864 15316 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:15.683919 15316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:15.684084 15316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:15.686085 15316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "0b8343ebfcfa45b08267cb624b6af83b"
format_stamp: "Formatted at 2026-06-19 18:47:15 on dist-test-slave-fljv"
server_key: "f9cfdfbc93a5933c07438a455896928d"
server_key_iv: "467a99686802c5098d1636203d38c1de"
server_key_version: "encryptionkey@0"
I20260619 18:47:15.686645 15316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "0b8343ebfcfa45b08267cb624b6af83b"
format_stamp: "Formatted at 2026-06-19 18:47:15 on dist-test-slave-fljv"
server_key: "f9cfdfbc93a5933c07438a455896928d"
server_key_iv: "467a99686802c5098d1636203d38c1de"
server_key_version: "encryptionkey@0"
I20260619 18:47:15.690340 15316 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:47:15.692833 15331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:15.694231 15316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:15.694357 15316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "0b8343ebfcfa45b08267cb624b6af83b"
format_stamp: "Formatted at 2026-06-19 18:47:15 on dist-test-slave-fljv"
server_key: "f9cfdfbc93a5933c07438a455896928d"
server_key_iv: "467a99686802c5098d1636203d38c1de"
server_key_version: "encryptionkey@0"
I20260619 18:47:15.694521 15316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:15.707690 15316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:15.710855 15316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:15.711010 15316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:15.712426 15316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:15.712477 15316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:15.712535 15316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:15.712565 15316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:15.722322 15316 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:33875
I20260619 18:47:15.722348 15446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:33875 every 8 connection(s)
Jun 19 18:47:15 dist-test-slave-fljv krb5kdc[14919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894835, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:15.723407 15316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:15.726751 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.713345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:39865 (local address 127.12.234.254:43431)
0619 18:47:15.713455 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.713459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.714578 (+ 1119us) server_negotiation.cc:407] Connection header received
0619 18:47:15.715693 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.715697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.715762 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.715857 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:15.717325 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.718157 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.719798 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.720061 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.720154 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.723207 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:15.723210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:15.723212 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:15.723240 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:15.724756 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.725273 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.725277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.725279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.725358 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.725659 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.725663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.725664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.725848 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:15.725932 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.726459 (+ 527us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.726578 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 18:47:15.727551 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.713905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43431 (local address 127.12.234.195:39865)
0619 18:47:15.714421 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.714465 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.715447 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.716045 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.716063 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.716460 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:15.717162 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.717182 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.718297 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.718301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.719673 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.719683 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.720205 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.720209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.720407 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.721093 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:15.721116 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:15.722940 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:15.724869 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.724874 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.724891 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.725171 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.725439 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.725442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.725444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.725577 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.725941 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:15.725947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:15.726220 (+ 273us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.726404 (+ 184us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.726745 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":123,"threads_started":1}
I20260619 18:47:15.728706 15447 heartbeater.cc:347] Connected to a master server at 127.12.234.254:43431
I20260619 18:47:15.728840 15447 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:15.729321 15447 heartbeater.cc:511] Master 127.12.234.254:43431 requested a full tablet report, sending...
I20260619 18:47:15.730283 14963 ts_manager.cc:194] Registered new tserver with Master: 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:15.730903 14963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:39865
I20260619 18:47:15.731782 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 15316
I20260619 18:47:15.731897 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:15.732143 13227 external_mini_cluster.cc:1428] Setting key d3e5f596b98fb9162d69a06f72bcb8a7
I20260619 18:47:15.734021 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:47:15.737485 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.730728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43431 (local address 127.12.234.195:41891)
0619 18:47:15.731056 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.731069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.731161 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.731356 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.731359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.731564 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:15.731736 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.731745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.732693 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.732697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.733432 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.733442 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.733837 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.733840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.733932 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.734443 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:15.734454 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:15.734756 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:15.736277 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.736280 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.736281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.736461 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.736724 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:15.736726 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:15.736728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:15.736792 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:15.737104 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:15.737106 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:15.737173 (+ 67us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.737250 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.737351 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260619 18:47:15.737669 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.730805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:41891 (local address 127.12.234.254:43431)
0619 18:47:15.730920 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.730924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.731133 (+ 209us) server_negotiation.cc:407] Connection header received
0619 18:47:15.731232 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.731235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.731274 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.731334 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:15.731858 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.732582 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.733525 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.733733 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.733822 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.734874 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:15.734878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:15.734880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:15.734904 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:15.736181 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.736540 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.736543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.736545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.736635 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.736860 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.736863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.736865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.737035 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:15.737111 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.737308 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.737413 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 18:47:15 dist-test-slave-fljv krb5kdc[14919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 18:47:15 dist-test-slave-fljv krb5kdc[14919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:15.760202 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.752147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.12.234.254:43431)
0619 18:47:15.752291 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.752295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.752308 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:47:15.752352 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.752354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.752399 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.752471 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:15.753282 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.753822 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.754782 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.755029 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.755111 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.757555 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:15.757558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:15.757560 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:15.757587 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:15.758912 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.759275 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.759277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.759279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.759351 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:15.759538 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:15.759540 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:15.759541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:15.759700 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:15.759818 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.759985 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.760080 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 18:47:15.762984 14963 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:15.765969 14963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:15.780354 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.775484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.12.234.194:33307)
0619 18:47:15.775776 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.775785 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.775804 (+ 19us) server_negotiation.cc:407] Connection header received
0619 18:47:15.775867 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.775872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.776058 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.776236 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:15.776401 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.777454 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.778689 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.779552 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.779696 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.779832 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.780099 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.780199 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":89,"threads_started":1}
I20260619 18:47:15.780359 15467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.774919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:33307 (local address 127.0.0.1:54132)
0619 18:47:15.775346 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.775357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.775427 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.776222 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.776225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.776242 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:15.776312 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.776317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.777568 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.777571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.778567 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.778577 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.779686 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.779690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.779847 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.779939 (+ 92us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.780053 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.780187 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":77,"thread_start_us":88,"threads_started":1}
I20260619 18:47:15.780359 15466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.774242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:33875 (local address 127.0.0.1:40068)
0619 18:47:15.775167 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.775179 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.775278 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.775943 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.775946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.775963 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:15.776055 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.776061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.777141 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.777145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.778718 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.778725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.779686 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.779690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.779867 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.779939 (+ 72us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.780053 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.780187 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":121,"thread_start_us":96,"threads_started":1}
I20260619 18:47:15.781287 15468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.775022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40068 (local address 127.12.234.195:33875)
0619 18:47:15.775544 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.775556 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.775578 (+ 22us) server_negotiation.cc:407] Connection header received
0619 18:47:15.775654 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.775659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.775829 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.775962 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:15.776144 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.777039 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.778823 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.779551 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.779698 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.779832 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.781016 (+ 1184us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.781131 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":93,"threads_started":1}
I20260619 18:47:15.781641 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.774082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:39741 (local address 127.0.0.1:46862)
0619 18:47:15.774689 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.774739 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.774866 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.775613 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.775619 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.775642 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:15.775774 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.775780 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.777162 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.777165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.778699 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.778710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.781251 (+ 2541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.781254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.781353 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.781378 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.781448 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.781518 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":234,"thread_start_us":59,"threads_started":1}
I20260619 18:47:15.782037 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.774245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.12.234.193:39741)
0619 18:47:15.774811 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.774828 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.774853 (+ 25us) server_negotiation.cc:407] Connection header received
0619 18:47:15.775061 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.775067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.775300 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.775446 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:15.775884 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.777051 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.779892 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.780922 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.781054 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.781178 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.781809 (+ 631us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.781904 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":86,"threads_started":1}
I20260619 18:47:15.783931 15228 tablet_service.cc:1467] Processing CreateTablet for tablet 842a8c1bd6d541baa7ba029dfabe58f6 (DEFAULT_TABLE table=test-table [id=9c9e94678dbb4cffb53281ec06a4fc33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:15.784062 15381 tablet_service.cc:1467] Processing CreateTablet for tablet 842a8c1bd6d541baa7ba029dfabe58f6 (DEFAULT_TABLE table=test-table [id=9c9e94678dbb4cffb53281ec06a4fc33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:15.784411 15101 tablet_service.cc:1467] Processing CreateTablet for tablet 842a8c1bd6d541baa7ba029dfabe58f6 (DEFAULT_TABLE table=test-table [id=9c9e94678dbb4cffb53281ec06a4fc33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:15.785017 15228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 842a8c1bd6d541baa7ba029dfabe58f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:15.785168 15381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 842a8c1bd6d541baa7ba029dfabe58f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:15.785264 15101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 842a8c1bd6d541baa7ba029dfabe58f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:15.791641 15470 tablet_bootstrap.cc:492] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: Bootstrap starting.
I20260619 18:47:15.792470 15471 tablet_bootstrap.cc:492] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: Bootstrap starting.
I20260619 18:47:15.793519 15472 tablet_bootstrap.cc:492] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: Bootstrap starting.
I20260619 18:47:15.794103 15470 tablet_bootstrap.cc:654] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:15.794648 15471 tablet_bootstrap.cc:654] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:15.795238 15470 log.cc:824] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:15.795404 15471 log.cc:824] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:15.795539 15472 tablet_bootstrap.cc:654] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:15.796344 15472 log.cc:824] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:15.797478 15470 tablet_bootstrap.cc:492] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: No bootstrap required, opened a new log
I20260619 18:47:15.797478 15471 tablet_bootstrap.cc:492] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: No bootstrap required, opened a new log
I20260619 18:47:15.797675 15471 ts_tablet_manager.cc:1397] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 18:47:15.797675 15470 ts_tablet_manager.cc:1397] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260619 18:47:15.797972 15472 tablet_bootstrap.cc:492] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: No bootstrap required, opened a new log
I20260619 18:47:15.798228 15472 ts_tablet_manager.cc:1397] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 18:47:15.801146 15472 raft_consensus.cc:348] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } }
I20260619 18:47:15.801381 15472 raft_consensus.cc:374] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:15.801450 15472 raft_consensus.cc:729] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8343ebfcfa45b08267cb624b6af83b, State: Initialized, Role: FOLLOWER
I20260619 18:47:15.801623 15471 raft_consensus.cc:348] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } }
I20260619 18:47:15.801623 15470 raft_consensus.cc:348] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } }
I20260619 18:47:15.801893 15471 raft_consensus.cc:374] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:15.801893 15470 raft_consensus.cc:374] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:15.801949 15470 raft_consensus.cc:729] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08edb91831b34a088ad1f11ec9d99dc4, State: Initialized, Role: FOLLOWER
I20260619 18:47:15.801976 15471 raft_consensus.cc:729] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940d425fb5ed45f48d81977dd94ac2c9, State: Initialized, Role: FOLLOWER
I20260619 18:47:15.801904 15472 consensus_queue.cc:260] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } }
I20260619 18:47:15.802469 15471 consensus_queue.cc:260] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } }
I20260619 18:47:15.802469 15470 consensus_queue.cc:260] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } }
I20260619 18:47:15.803078 15472 ts_tablet_manager.cc:1428] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260619 18:47:15.803432 15447 heartbeater.cc:503] Master 127.12.234.254:43431 was elected leader, sending a full tablet report...
I20260619 18:47:15.803483 15471 ts_tablet_manager.cc:1428] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260619 18:47:15.803483 15470 ts_tablet_manager.cc:1428] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:47:15.803812 15307 heartbeater.cc:503] Master 127.12.234.254:43431 was elected leader, sending a full tablet report...
I20260619 18:47:15.804708 15167 heartbeater.cc:503] Master 127.12.234.254:43431 was elected leader, sending a full tablet report...
I20260619 18:47:15.951992 15477 raft_consensus.cc:482] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:15.952247 15477 raft_consensus.cc:504] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } }
I20260619 18:47:15.953510 15477 leader_election.cc:288] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:15.957759 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.953805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:39741 (local address 127.12.234.194:45665)
0619 18:47:15.953905 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.953920 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.954021 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.954290 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.954293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.954312 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:15.954435 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.954441 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.955393 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.955396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.956630 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.956642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.957385 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.957389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.957479 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.957501 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.957572 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.957636 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 18:47:15.957798 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.953885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:45665 (local address 127.12.234.193:39741)
0619 18:47:15.954005 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.954010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.954026 (+ 16us) server_negotiation.cc:407] Connection header received
0619 18:47:15.954114 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.954116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.954156 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.954262 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:15.954522 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.955271 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.956780 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.957232 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.957370 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.957424 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.957598 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.957672 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:47:15.958601 15468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.953885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:46919 (local address 127.12.234.195:33875)
0619 18:47:15.954005 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:15.954009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:15.954397 (+ 388us) server_negotiation.cc:407] Connection header received
0619 18:47:15.954514 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:15.954516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:15.954565 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:15.954641 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:15.954856 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.955535 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.957254 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.957824 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.957940 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.957989 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:15.958386 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 18:47:15.958473 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 18:47:15.958604 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "842a8c1bd6d541baa7ba029dfabe58f6" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940d425fb5ed45f48d81977dd94ac2c9" is_pre_election: true
I20260619 18:47:15.958940 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:15.953805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:33875 (local address 127.12.234.194:46919)
0619 18:47:15.954308 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:15.954321 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:15.954429 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:15.954652 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:15.954654 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:15.954669 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:15.954760 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.954765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.955649 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.955652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.957110 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:15.957119 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:15.958078 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:15.958082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:15.958239 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:15.958265 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 18:47:15.958748 (+ 483us) client_negotiation.cc:240] Negotiation successful
0619 18:47:15.958827 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":109,"thread_start_us":117,"threads_started":1}
I20260619 18:47:15.958952 15121 raft_consensus.cc:2454] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 0.
I20260619 18:47:15.959517 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "842a8c1bd6d541baa7ba029dfabe58f6" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b" is_pre_election: true
I20260619 18:47:15.959538 15195 leader_election.cc:302] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08edb91831b34a088ad1f11ec9d99dc4, 940d425fb5ed45f48d81977dd94ac2c9; no voters:
I20260619 18:47:15.959789 15401 raft_consensus.cc:2454] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 0.
I20260619 18:47:15.959790 15477 raft_consensus.cc:2781] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:15.959864 15477 raft_consensus.cc:482] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:15.959893 15477 raft_consensus.cc:3037] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:15.961155 15477 raft_consensus.cc:504] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } }
I20260619 18:47:15.961503 15477 leader_election.cc:288] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 election: Requested vote from peers 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:15.961828 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "842a8c1bd6d541baa7ba029dfabe58f6" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940d425fb5ed45f48d81977dd94ac2c9"
I20260619 18:47:15.961897 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "842a8c1bd6d541baa7ba029dfabe58f6" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b"
I20260619 18:47:15.961952 15121 raft_consensus.cc:3037] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:15.962036 15401 raft_consensus.cc:3037] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:15.963011 15121 raft_consensus.cc:2454] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 1.
I20260619 18:47:15.963116 15401 raft_consensus.cc:2454] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 1.
I20260619 18:47:15.963371 15195 leader_election.cc:302] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08edb91831b34a088ad1f11ec9d99dc4, 940d425fb5ed45f48d81977dd94ac2c9; no voters:
I20260619 18:47:15.963549 15477 raft_consensus.cc:2781] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:15.963765 15477 raft_consensus.cc:686] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 LEADER]: Becoming Leader. State: Replica: 08edb91831b34a088ad1f11ec9d99dc4, State: Running, Role: LEADER
I20260619 18:47:15.964112 15477 consensus_queue.cc:237] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } }
I20260619 18:47:15.967528 14963 catalog_manager.cc:5495] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194). New cstate: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } health_report { overall_health: HEALTHY } } }
W20260619 18:47:15.974821 15448 tablet.cc:2369] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:47:16.015472 15308 tablet.cc:2369] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 18:47:16 dist-test-slave-fljv krb5kdc[14919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894834, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:16.036705 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:16.027950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43431 (local address 127.0.0.1:44256)
0619 18:47:16.028256 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:16.028271 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:16.028387 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:16.028622 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:16.028624 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:16.029053 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:16.029215 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:16.029222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.029964 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.029967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:16.030604 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:16.030613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.031070 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.031073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:16.031150 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:16.031683 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:16.031694 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:16.033232 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:16.035022 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:16.035033 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:16.035041 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:16.035334 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:16.035606 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:16.035609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:16.035611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:16.035697 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:16.036047 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:16.036056 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:16.036209 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 18:47:16.036315 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 18:47:16.036457 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":129,"threads_started":1}
I20260619 18:47:16.036713 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:16.028031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.12.234.254:43431)
0619 18:47:16.028212 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:16.028215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:16.028369 (+ 154us) server_negotiation.cc:407] Connection header received
0619 18:47:16.028463 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:16.028466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:16.028516 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:16.028596 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:16.029342 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.029854 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.030727 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.030939 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.031037 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:16.033389 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:16.033393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:16.033395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:16.033418 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:16.034918 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:16.035425 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:16.035428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:16.035429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:16.035506 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:16.035767 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:16.035769 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:16.035770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:16.035939 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:16.036008 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:16.036360 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 18:47:16.036533 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":98}
W20260619 18:47:16.040568 15168 tablet.cc:2369] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:47:16.040527 14963 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:44256:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 18:47:16.041273 14963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:16.045925 15101 tablet_service.cc:1467] Processing CreateTablet for tablet be2d8815c9244b8aaf483f79ec08281a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=643cbc8e619441688c10a194382a524d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 18:47:16.045979 15228 tablet_service.cc:1467] Processing CreateTablet for tablet be2d8815c9244b8aaf483f79ec08281a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=643cbc8e619441688c10a194382a524d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 18:47:16.046298 15381 tablet_service.cc:1467] Processing CreateTablet for tablet be2d8815c9244b8aaf483f79ec08281a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=643cbc8e619441688c10a194382a524d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 18:47:16.046734 15101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2d8815c9244b8aaf483f79ec08281a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:16.046756 15228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2d8815c9244b8aaf483f79ec08281a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:16.046824 15381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2d8815c9244b8aaf483f79ec08281a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:16.049136 15471 tablet_bootstrap.cc:492] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: Bootstrap starting.
I20260619 18:47:16.049578 15472 tablet_bootstrap.cc:492] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: Bootstrap starting.
I20260619 18:47:16.049881 15470 tablet_bootstrap.cc:492] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: Bootstrap starting.
I20260619 18:47:16.050148 15471 tablet_bootstrap.cc:654] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:16.050576 15472 tablet_bootstrap.cc:654] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:16.050863 15470 tablet_bootstrap.cc:654] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:16.051482 15471 tablet_bootstrap.cc:492] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: No bootstrap required, opened a new log
I20260619 18:47:16.051589 15471 ts_tablet_manager.cc:1397] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 18:47:16.052022 15470 tablet_bootstrap.cc:492] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: No bootstrap required, opened a new log
I20260619 18:47:16.052110 15470 ts_tablet_manager.cc:1397] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:16.052049 15471 raft_consensus.cc:348] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.052156 15471 raft_consensus.cc:374] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:16.052196 15471 raft_consensus.cc:729] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940d425fb5ed45f48d81977dd94ac2c9, State: Initialized, Role: FOLLOWER
I20260619 18:47:16.052315 15471 consensus_queue.cc:260] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.052553 15470 raft_consensus.cc:348] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.052590 15478 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.052671 15470 raft_consensus.cc:374] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:16.052701 15470 raft_consensus.cc:729] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08edb91831b34a088ad1f11ec9d99dc4, State: Initialized, Role: FOLLOWER
I20260619 18:47:16.052722 15478 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:16.052814 15470 consensus_queue.cc:260] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.053077 15470 ts_tablet_manager.cc:1428] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:16.053077 15471 ts_tablet_manager.cc:1428] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 18:47:16.053051 15477 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.053175 15477 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:16.053902 15472 tablet_bootstrap.cc:492] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: No bootstrap required, opened a new log
I20260619 18:47:16.054018 15472 ts_tablet_manager.cc:1397] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260619 18:47:16.054522 15472 raft_consensus.cc:348] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.054642 15472 raft_consensus.cc:374] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:16.054677 15472 raft_consensus.cc:729] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8343ebfcfa45b08267cb624b6af83b, State: Initialized, Role: FOLLOWER
I20260619 18:47:16.054791 15472 consensus_queue.cc:260] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.055076 15472 ts_tablet_manager.cc:1428] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:16.055097 15476 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.055248 15476 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:16.092964 15477 raft_consensus.cc:482] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:16.093102 15477 raft_consensus.cc:504] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.093477 15477 leader_election.cc:288] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:16.093915 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2d8815c9244b8aaf483f79ec08281a" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b" is_pre_election: true
I20260619 18:47:16.093920 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2d8815c9244b8aaf483f79ec08281a" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940d425fb5ed45f48d81977dd94ac2c9" is_pre_election: true
I20260619 18:47:16.094120 15401 raft_consensus.cc:2454] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 0.
I20260619 18:47:16.094125 15121 raft_consensus.cc:2454] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 0.
I20260619 18:47:16.094434 15192 leader_election.cc:302] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08edb91831b34a088ad1f11ec9d99dc4, 0b8343ebfcfa45b08267cb624b6af83b; no voters:
I20260619 18:47:16.094640 15477 raft_consensus.cc:2781] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:16.094696 15477 raft_consensus.cc:482] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:16.094722 15477 raft_consensus.cc:3037] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:16.095736 15477 raft_consensus.cc:504] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.096076 15477 leader_election.cc:288] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 election: Requested vote from peers 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:16.096403 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2d8815c9244b8aaf483f79ec08281a" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940d425fb5ed45f48d81977dd94ac2c9"
I20260619 18:47:16.096511 15121 raft_consensus.cc:3037] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:16.096490 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2d8815c9244b8aaf483f79ec08281a" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b"
I20260619 18:47:16.096657 15401 raft_consensus.cc:3037] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:16.097429 15121 raft_consensus.cc:2454] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 1.
I20260619 18:47:16.097642 15401 raft_consensus.cc:2454] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 1.
I20260619 18:47:16.097692 15195 leader_election.cc:302] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08edb91831b34a088ad1f11ec9d99dc4, 940d425fb5ed45f48d81977dd94ac2c9; no voters:
I20260619 18:47:16.097849 15477 raft_consensus.cc:2781] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:16.097922 15477 raft_consensus.cc:686] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 LEADER]: Becoming Leader. State: Replica: 08edb91831b34a088ad1f11ec9d99dc4, State: Running, Role: LEADER
I20260619 18:47:16.098057 15477 consensus_queue.cc:237] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.098759 15480 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: TxnStatusTablet state changed. Reason: New leader 08edb91831b34a088ad1f11ec9d99dc4. Latest consensus state: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.098873 15480 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: This TxnStatusTablet replica's current role is: LEADER
I20260619 18:47:16.099128 15492 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 18:47:16.099349 15492 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 18:47:16.099864 14963 catalog_manager.cc:5495] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194). New cstate: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } health_report { overall_health: UNKNOWN } } }
I20260619 18:47:16.113178 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:16.108873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:33307 (local address 127.0.0.1:54140)
0619 18:47:16.109026 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:16.109042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:16.109147 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:16.109518 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:16.109521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:16.109529 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:16.109627 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:16.109632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.110671 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.110674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:16.111336 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:16.111346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.111929 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.111932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:16.112002 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:16.112051 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:16.112873 (+ 822us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:16.112887 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 18:47:16.112937 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 18:47:16.113021 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 18:47:16.113168 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:16.108998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54140 (local address 127.12.234.194:33307)
0619 18:47:16.109132 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:16.109136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:16.109153 (+ 17us) server_negotiation.cc:407] Connection header received
0619 18:47:16.109279 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:16.109283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:16.109381 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:16.109492 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:16.109726 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.110554 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.111461 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.111817 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.111922 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:16.112221 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:16.112774 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:16.112888 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:16.112952 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 18:47:16.113015 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 18:47:16.118494 15121 raft_consensus.cc:1261] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Refusing update from remote peer 08edb91831b34a088ad1f11ec9d99dc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:16.118499 15401 raft_consensus.cc:1261] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Refusing update from remote peer 08edb91831b34a088ad1f11ec9d99dc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:16.119211 15480 consensus_queue.cc:1035] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:16.119393 15477 consensus_queue.cc:1035] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:16.132616 15478 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: TxnStatusTablet state changed. Reason: New leader 08edb91831b34a088ad1f11ec9d99dc4. Latest consensus state: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.132815 15478 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:16.134358 15476 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: TxnStatusTablet state changed. Reason: New leader 08edb91831b34a088ad1f11ec9d99dc4. Latest consensus state: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.134493 15476 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:16.135190 15480 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.135314 15480 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: This TxnStatusTablet replica's current role is: LEADER
I20260619 18:47:16.135288 15495 mvcc.cc:205] Tried to move back new op lower bound from 7298641248734457856 to 7298641248659202048. Current Snapshot: MvccSnapshot[applied={T|T < 7298641248734457856}]
I20260619 18:47:16.135200 15477 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.135953 15477 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 08edb91831b34a088ad1f11ec9d99dc4: This TxnStatusTablet replica's current role is: LEADER
I20260619 18:47:16.137243 15476 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.137390 15476 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 0b8343ebfcfa45b08267cb624b6af83b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:16.138644 15478 tablet_replica.cc:439] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } } }
I20260619 18:47:16.138772 15478 tablet_replica.cc:442] T be2d8815c9244b8aaf483f79ec08281a P 940d425fb5ed45f48d81977dd94ac2c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:16.139503 15496 mvcc.cc:205] Tried to move back new op lower bound from 7298641248734457856 to 7298641248659202048. Current Snapshot: MvccSnapshot[applied={T|T < 7298641248734457856}]
I20260619 18:47:16.154438 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:16.150973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.12.234.194:33307)
0619 18:47:16.151458 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:16.151462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:16.151473 (+ 11us) server_negotiation.cc:407] Connection header received
0619 18:47:16.151513 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:16.151515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:16.151576 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:16.151643 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:16.151902 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.152542 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.153406 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.153634 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.153723 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:16.153979 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:16.154066 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:16.154122 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:16.154245 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 18:47:16.154298 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394}
I20260619 18:47:16.165170 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:16.162029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:53937 (local address 127.12.234.194:33307)
0619 18:47:16.162320 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:16.162324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:16.162354 (+ 30us) server_negotiation.cc:407] Connection header received
0619 18:47:16.162494 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:16.162497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:16.162538 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:16.162595 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:16.162840 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.163332 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.164150 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.164366 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.164447 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:16.164741 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:16.164812 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:16.164868 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:16.164951 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 18:47:16.165009 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":238}
I20260619 18:47:16.165176 15507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:16.161995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:33307 (local address 127.12.234.194:53937)
0619 18:47:16.162287 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:16.162301 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:16.162409 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:16.162610 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:16.162613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:16.162619 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:16.162755 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:16.162760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.163429 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.163432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:16.164049 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:16.164057 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:16.164474 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:16.164477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:16.164566 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:16.164606 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:16.164882 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:16.164888 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:47:16.164944 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 18:47:16.165046 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":36,"thread_start_us":99,"threads_started":1}
I20260619 18:47:16.172851 15121 raft_consensus.cc:1261] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Refusing update from remote peer 08edb91831b34a088ad1f11ec9d99dc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:16.173255 15508 consensus_queue.cc:1035] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:16.173385 15401 raft_consensus.cc:1261] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Refusing update from remote peer 08edb91831b34a088ad1f11ec9d99dc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:16.176136 15508 consensus_queue.cc:1035] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:16.179023 15497 mvcc.cc:205] Tried to move back new op lower bound from 7298641248957923328 to 7298641248111763456. Current Snapshot: MvccSnapshot[applied={T|T < 7298641248957923328}]
I20260619 18:47:16.236075 14963 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:16.236250 14963 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:16.238665 14963 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc: Sending DeleteTablet for 3 replicas of tablet 842a8c1bd6d541baa7ba029dfabe58f6
I20260619 18:47:16.239401 15101 tablet_service.cc:1514] Processing DeleteTablet for tablet 842a8c1bd6d541baa7ba029dfabe58f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:16 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:46862
I20260619 18:47:16.239665 15381 tablet_service.cc:1514] Processing DeleteTablet for tablet 842a8c1bd6d541baa7ba029dfabe58f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:16 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:40068
I20260619 18:47:16.240069 15514 tablet_replica.cc:330] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: stopping tablet replica
I20260619 18:47:16.240475 15514 raft_consensus.cc:2229] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:16.240710 15228 tablet_service.cc:1514] Processing DeleteTablet for tablet 842a8c1bd6d541baa7ba029dfabe58f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:16 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:54132
I20260619 18:47:16.240787 15514 raft_consensus.cc:2258] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:16.241457 15513 tablet_replica.cc:330] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: stopping tablet replica
I20260619 18:47:16.241847 15513 raft_consensus.cc:2229] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:16.241854 14963 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:16.242328 14963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:16.242376 15513 raft_consensus.cc:2258] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:16.244036 15513 ts_tablet_manager.cc:1905] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:16.247509 15513 ts_tablet_manager.cc:1918] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:16.247711 15513 log.cc:1196] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/842a8c1bd6d541baa7ba029dfabe58f6
I20260619 18:47:16.247972 15101 tablet_service.cc:1467] Processing CreateTablet for tablet afde5354403446d3ba064dbb59f34dc2 (DEFAULT_TABLE table=test-table [id=79ff0859a71545a09f6ced654e1a9ce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:16.248227 15101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afde5354403446d3ba064dbb59f34dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:16.248992 15381 tablet_service.cc:1467] Processing CreateTablet for tablet afde5354403446d3ba064dbb59f34dc2 (DEFAULT_TABLE table=test-table [id=79ff0859a71545a09f6ced654e1a9ce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:16.249251 15381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afde5354403446d3ba064dbb59f34dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:16.250604 15471 tablet_bootstrap.cc:492] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: Bootstrap starting.
I20260619 18:47:16.251541 15471 tablet_bootstrap.cc:654] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:16.251580 15472 tablet_bootstrap.cc:492] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: Bootstrap starting.
I20260619 18:47:16.248227 15513 ts_tablet_manager.cc:1939] T 842a8c1bd6d541baa7ba029dfabe58f6 P 940d425fb5ed45f48d81977dd94ac2c9: Deleting consensus metadata
I20260619 18:47:16.252439 15472 tablet_bootstrap.cc:654] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:16.253759 14951 catalog_manager.cc:4841] TS 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741): tablet 842a8c1bd6d541baa7ba029dfabe58f6 (table test-table [id=9c9e94678dbb4cffb53281ec06a4fc33]) successfully deleted
I20260619 18:47:16.253854 15228 tablet_service.cc:1467] Processing CreateTablet for tablet afde5354403446d3ba064dbb59f34dc2 (DEFAULT_TABLE table=test-table [id=79ff0859a71545a09f6ced654e1a9ce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:16.254146 15228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afde5354403446d3ba064dbb59f34dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:16.254388 15515 tablet_replica.cc:330] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: stopping tablet replica
I20260619 18:47:16.254737 15515 raft_consensus.cc:2229] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:16.255146 15515 raft_consensus.cc:2258] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:16.256600 15515 ts_tablet_manager.cc:1905] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:16.256959 15470 tablet_bootstrap.cc:492] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: Bootstrap starting.
I20260619 18:47:16.257783 15470 tablet_bootstrap.cc:654] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:16.259021 15470 tablet_bootstrap.cc:492] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: No bootstrap required, opened a new log
I20260619 18:47:16.259104 15470 ts_tablet_manager.cc:1397] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 18:47:16.259483 15470 raft_consensus.cc:348] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.259588 15470 raft_consensus.cc:374] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:16.259617 15470 raft_consensus.cc:729] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08edb91831b34a088ad1f11ec9d99dc4, State: Initialized, Role: FOLLOWER
I20260619 18:47:16.259737 15470 consensus_queue.cc:260] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.259963 15515 ts_tablet_manager.cc:1918] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:16.259971 15470 ts_tablet_manager.cc:1428] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 18:47:16.260082 15515 log.cc:1196] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/842a8c1bd6d541baa7ba029dfabe58f6
I20260619 18:47:16.260466 15515 ts_tablet_manager.cc:1939] T 842a8c1bd6d541baa7ba029dfabe58f6 P 08edb91831b34a088ad1f11ec9d99dc4: Deleting consensus metadata
I20260619 18:47:16.261672 14950 catalog_manager.cc:4841] TS 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194:33307): tablet 842a8c1bd6d541baa7ba029dfabe58f6 (table test-table [id=9c9e94678dbb4cffb53281ec06a4fc33]) successfully deleted
I20260619 18:47:16.262475 15514 ts_tablet_manager.cc:1905] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:16.265383 15514 ts_tablet_manager.cc:1918] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:16.265482 15514 log.cc:1196] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/842a8c1bd6d541baa7ba029dfabe58f6
I20260619 18:47:16.265743 15514 ts_tablet_manager.cc:1939] T 842a8c1bd6d541baa7ba029dfabe58f6 P 0b8343ebfcfa45b08267cb624b6af83b: Deleting consensus metadata
I20260619 18:47:16.267153 14948 catalog_manager.cc:4841] TS 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875): tablet 842a8c1bd6d541baa7ba029dfabe58f6 (table test-table [id=9c9e94678dbb4cffb53281ec06a4fc33]) successfully deleted
I20260619 18:47:16.267652 15472 tablet_bootstrap.cc:492] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: No bootstrap required, opened a new log
I20260619 18:47:16.267753 15472 ts_tablet_manager.cc:1397] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20260619 18:47:16.268244 15472 raft_consensus.cc:348] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.268366 15472 raft_consensus.cc:374] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:16.268397 15472 raft_consensus.cc:729] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8343ebfcfa45b08267cb624b6af83b, State: Initialized, Role: FOLLOWER
I20260619 18:47:16.268533 15472 consensus_queue.cc:260] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.268774 15472 ts_tablet_manager.cc:1428] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:16.270107 15471 tablet_bootstrap.cc:492] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: No bootstrap required, opened a new log
I20260619 18:47:16.270233 15471 ts_tablet_manager.cc:1397] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20260619 18:47:16.270725 15471 raft_consensus.cc:348] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.270833 15471 raft_consensus.cc:374] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:16.270864 15471 raft_consensus.cc:729] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940d425fb5ed45f48d81977dd94ac2c9, State: Initialized, Role: FOLLOWER
I20260619 18:47:16.270987 15471 consensus_queue.cc:260] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.271224 15471 ts_tablet_manager.cc:1428] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:16.285175 15501 raft_consensus.cc:482] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:16.285300 15501 raft_consensus.cc:504] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.285759 15501 leader_election.cc:288] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:16.286157 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afde5354403446d3ba064dbb59f34dc2" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940d425fb5ed45f48d81977dd94ac2c9" is_pre_election: true
I20260619 18:47:16.286258 15400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afde5354403446d3ba064dbb59f34dc2" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b" is_pre_election: true
I20260619 18:47:16.286362 15121 raft_consensus.cc:2454] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 0.
I20260619 18:47:16.286374 15400 raft_consensus.cc:2454] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 0.
I20260619 18:47:16.286711 15195 leader_election.cc:302] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08edb91831b34a088ad1f11ec9d99dc4, 0b8343ebfcfa45b08267cb624b6af83b, 940d425fb5ed45f48d81977dd94ac2c9; no voters:
I20260619 18:47:16.286960 15501 raft_consensus.cc:2781] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:16.287014 15501 raft_consensus.cc:482] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:16.287055 15501 raft_consensus.cc:3037] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:16.288084 15501 raft_consensus.cc:504] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.288441 15501 leader_election.cc:288] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 election: Requested vote from peers 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:16.293812 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afde5354403446d3ba064dbb59f34dc2" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940d425fb5ed45f48d81977dd94ac2c9"
I20260619 18:47:16.293819 15400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afde5354403446d3ba064dbb59f34dc2" candidate_uuid: "08edb91831b34a088ad1f11ec9d99dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b"
I20260619 18:47:16.293947 15121 raft_consensus.cc:3037] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:16.293954 15400 raft_consensus.cc:3037] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:16.295125 15121 raft_consensus.cc:2454] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 1.
I20260619 18:47:16.295128 15400 raft_consensus.cc:2454] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08edb91831b34a088ad1f11ec9d99dc4 in term 1.
I20260619 18:47:16.295514 15192 leader_election.cc:302] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08edb91831b34a088ad1f11ec9d99dc4, 0b8343ebfcfa45b08267cb624b6af83b; no voters:
I20260619 18:47:16.295780 15501 raft_consensus.cc:2781] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:16.295859 15501 raft_consensus.cc:686] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 LEADER]: Becoming Leader. State: Replica: 08edb91831b34a088ad1f11ec9d99dc4, State: Running, Role: LEADER
I20260619 18:47:16.295977 15501 consensus_queue.cc:237] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:16.297734 14963 catalog_manager.cc:5495] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194). New cstate: current_term: 1 leader_uuid: "08edb91831b34a088ad1f11ec9d99dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } health_report { overall_health: UNKNOWN } } }
I20260619 18:47:16.326509 15121 raft_consensus.cc:1261] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Refusing update from remote peer 08edb91831b34a088ad1f11ec9d99dc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:16.326891 15480 consensus_queue.cc:1035] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:16.326956 15400 raft_consensus.cc:1261] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Refusing update from remote peer 08edb91831b34a088ad1f11ec9d99dc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:16.327374 15480 consensus_queue.cc:1035] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 18:47:16.349152 15495 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 18:47:17.905248 14963 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:17.905406 14963 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:17.908031 14963 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc: Sending DeleteTablet for 3 replicas of tablet afde5354403446d3ba064dbb59f34dc2
I20260619 18:47:17.908608 15228 tablet_service.cc:1514] Processing DeleteTablet for tablet afde5354403446d3ba064dbb59f34dc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:17 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:54132
I20260619 18:47:17.908643 15101 tablet_service.cc:1514] Processing DeleteTablet for tablet afde5354403446d3ba064dbb59f34dc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:17 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:46862
I20260619 18:47:17.908929 15381 tablet_service.cc:1514] Processing DeleteTablet for tablet afde5354403446d3ba064dbb59f34dc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:17 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:40068
I20260619 18:47:17.908986 15543 tablet_replica.cc:330] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: stopping tablet replica
I20260619 18:47:17.909116 15543 raft_consensus.cc:2229] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:17.909312 15543 raft_consensus.cc:2258] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:17.909385 15544 tablet_replica.cc:330] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: stopping tablet replica
I20260619 18:47:17.909570 15544 raft_consensus.cc:2229] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:17.909698 15544 raft_consensus.cc:2258] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:17.909946 15543 ts_tablet_manager.cc:1905] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:17.910310 15544 ts_tablet_manager.cc:1905] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:17.912400 15544 ts_tablet_manager.cc:1918] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 18:47:17.912473 15544 log.cc:1196] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/afde5354403446d3ba064dbb59f34dc2
I20260619 18:47:17.912698 15544 ts_tablet_manager.cc:1939] T afde5354403446d3ba064dbb59f34dc2 P 0b8343ebfcfa45b08267cb624b6af83b: Deleting consensus metadata
I20260619 18:47:17.912810 15543 ts_tablet_manager.cc:1918] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 18:47:17.912909 15543 log.cc:1196] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/afde5354403446d3ba064dbb59f34dc2
I20260619 18:47:17.913231 15543 ts_tablet_manager.cc:1939] T afde5354403446d3ba064dbb59f34dc2 P 08edb91831b34a088ad1f11ec9d99dc4: Deleting consensus metadata
I20260619 18:47:17.913671 14948 catalog_manager.cc:4841] TS 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875): tablet afde5354403446d3ba064dbb59f34dc2 (table test-table [id=79ff0859a71545a09f6ced654e1a9ce8]) successfully deleted
I20260619 18:47:17.913853 15542 tablet_replica.cc:330] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: stopping tablet replica
I20260619 18:47:17.913928 14950 catalog_manager.cc:4841] TS 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194:33307): tablet afde5354403446d3ba064dbb59f34dc2 (table test-table [id=79ff0859a71545a09f6ced654e1a9ce8]) successfully deleted
I20260619 18:47:17.913990 15542 raft_consensus.cc:2229] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:17.914132 15542 raft_consensus.cc:2258] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:17.914745 15542 ts_tablet_manager.cc:1905] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:17.917128 15542 ts_tablet_manager.cc:1918] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 18:47:17.917215 15542 log.cc:1196] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/afde5354403446d3ba064dbb59f34dc2
I20260619 18:47:17.917416 15542 ts_tablet_manager.cc:1939] T afde5354403446d3ba064dbb59f34dc2 P 940d425fb5ed45f48d81977dd94ac2c9: Deleting consensus metadata
I20260619 18:47:17.918119 14951 catalog_manager.cc:4841] TS 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741): tablet afde5354403446d3ba064dbb59f34dc2 (table test-table [id=79ff0859a71545a09f6ced654e1a9ce8]) successfully deleted
Jun 19 18:47:17 dist-test-slave-fljv krb5kdc[14919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 18:47:17 dist-test-slave-fljv krb5kdc[14919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:17.938337 15552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:17.928740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44258 (local address 127.12.234.254:43431)
0619 18:47:17.929110 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:17.929119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:17.929177 (+ 58us) server_negotiation.cc:407] Connection header received
0619 18:47:17.929339 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:17.929342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:17.929398 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:17.929473 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:17.930356 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:17.930916 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:17.931748 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:17.931978 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:17.932065 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:17.934579 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:17.934587 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:17.934590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:17.934620 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:17.936188 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:17.936695 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:17.936699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:17.936700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:17.936792 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:17.937204 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:17.937207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:17.937209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:17.937417 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:17.937561 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:17.937978 (+ 417us) server_negotiation.cc:299] Negotiation successful
0619 18:47:17.938121 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":132,"threads_started":1}
I20260619 18:47:17.941185 14963 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:17.941628 14963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:17.946321 15101 tablet_service.cc:1467] Processing CreateTablet for tablet 3e15df85b09045b187f19a534b7284c8 (DEFAULT_TABLE table=test-table [id=22ec520903264571bc5b18c3b9f2bd3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:17.946426 15228 tablet_service.cc:1467] Processing CreateTablet for tablet 3e15df85b09045b187f19a534b7284c8 (DEFAULT_TABLE table=test-table [id=22ec520903264571bc5b18c3b9f2bd3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:17.946470 15381 tablet_service.cc:1467] Processing CreateTablet for tablet 3e15df85b09045b187f19a534b7284c8 (DEFAULT_TABLE table=test-table [id=22ec520903264571bc5b18c3b9f2bd3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:17.946643 15101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e15df85b09045b187f19a534b7284c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:17.946715 15228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e15df85b09045b187f19a534b7284c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:17.946750 15381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e15df85b09045b187f19a534b7284c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:17.948817 15553 tablet_bootstrap.cc:492] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: Bootstrap starting.
I20260619 18:47:17.949407 15555 tablet_bootstrap.cc:492] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: Bootstrap starting.
I20260619 18:47:17.949766 15553 tablet_bootstrap.cc:654] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:17.949838 15554 tablet_bootstrap.cc:492] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: Bootstrap starting.
I20260619 18:47:17.950379 15555 tablet_bootstrap.cc:654] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:17.950881 15554 tablet_bootstrap.cc:654] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:17.951027 15553 tablet_bootstrap.cc:492] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: No bootstrap required, opened a new log
I20260619 18:47:17.951122 15553 ts_tablet_manager.cc:1397] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:17.951529 15553 raft_consensus.cc:348] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:17.951651 15553 raft_consensus.cc:374] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:17.951712 15553 raft_consensus.cc:729] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08edb91831b34a088ad1f11ec9d99dc4, State: Initialized, Role: FOLLOWER
I20260619 18:47:17.951849 15553 consensus_queue.cc:260] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:17.952173 15553 ts_tablet_manager.cc:1428] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:17.952414 15554 tablet_bootstrap.cc:492] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: No bootstrap required, opened a new log
I20260619 18:47:17.952502 15555 tablet_bootstrap.cc:492] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: No bootstrap required, opened a new log
I20260619 18:47:17.952538 15554 ts_tablet_manager.cc:1397] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:17.952580 15555 ts_tablet_manager.cc:1397] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:17.952983 15555 raft_consensus.cc:348] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:17.952975 15554 raft_consensus.cc:348] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:17.953076 15554 raft_consensus.cc:374] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:17.953076 15555 raft_consensus.cc:374] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:17.953106 15555 raft_consensus.cc:729] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940d425fb5ed45f48d81977dd94ac2c9, State: Initialized, Role: FOLLOWER
I20260619 18:47:17.953106 15554 raft_consensus.cc:729] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8343ebfcfa45b08267cb624b6af83b, State: Initialized, Role: FOLLOWER
I20260619 18:47:17.953222 15555 consensus_queue.cc:260] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:17.953222 15554 consensus_queue.cc:260] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:17.953585 15555 ts_tablet_manager.cc:1428] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:17.953585 15554 ts_tablet_manager.cc:1428] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:18.092927 15559 raft_consensus.cc:482] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:18.093135 15559 raft_consensus.cc:504] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.094272 15559 leader_election.cc:288] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194:33307), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:18.098239 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:18.094556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:55335 (local address 127.12.234.194:33307)
0619 18:47:18.094958 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:18.094969 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:18.095005 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:47:18.095243 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:18.095245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:18.095286 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:18.095355 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:18.095651 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.096168 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.097215 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.097643 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.097747 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:18.097805 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:18.098004 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 18:47:18.098076 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":124,"threads_started":1}
I20260619 18:47:18.098258 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:18.094480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:33307 (local address 127.12.234.193:55335)
0619 18:47:18.094929 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:18.094944 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:18.095129 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:18.095371 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:18.095374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:18.095389 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:18.095547 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:18.095553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.096269 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.096272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:18.097097 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:18.097106 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.097747 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.097751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:18.097871 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:18.097914 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 18:47:18.097978 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 18:47:18.098052 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":21,"spinlock_wait_cycles":5376,"thread_start_us":199,"threads_started":1}
I20260619 18:47:18.098282 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:18.094506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:33875 (local address 127.12.234.193:38379)
0619 18:47:18.094930 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:18.094944 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:18.095133 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:18.095379 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:18.095382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:18.095398 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:18.095563 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:18.095568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.096359 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.096363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:18.097132 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:18.097142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.097797 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.097801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:18.097920 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:18.097942 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:47:18.098004 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 18:47:18.098074 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":36,"spinlock_wait_cycles":148864,"thread_start_us":205,"threads_started":1}
I20260619 18:47:18.098299 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:18.094609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:38379 (local address 127.12.234.195:33875)
0619 18:47:18.094917 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:18.094931 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:18.095006 (+ 75us) server_negotiation.cc:407] Connection header received
0619 18:47:18.095239 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:18.095242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:18.095293 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:18.095365 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:18.095663 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.096232 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.097232 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.097695 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.097797 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:18.097847 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:18.098038 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 18:47:18.098107 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":110,"threads_started":1}
I20260619 18:47:18.098711 15400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e15df85b09045b187f19a534b7284c8" candidate_uuid: "940d425fb5ed45f48d81977dd94ac2c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b" is_pre_election: true
I20260619 18:47:18.098861 15400 raft_consensus.cc:2454] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 940d425fb5ed45f48d81977dd94ac2c9 in term 0.
I20260619 18:47:18.099226 15052 leader_election.cc:302] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b8343ebfcfa45b08267cb624b6af83b, 940d425fb5ed45f48d81977dd94ac2c9; no voters:
I20260619 18:47:18.099239 15261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e15df85b09045b187f19a534b7284c8" candidate_uuid: "940d425fb5ed45f48d81977dd94ac2c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08edb91831b34a088ad1f11ec9d99dc4" is_pre_election: true
I20260619 18:47:18.099462 15261 raft_consensus.cc:2454] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 940d425fb5ed45f48d81977dd94ac2c9 in term 0.
I20260619 18:47:18.099509 15559 raft_consensus.cc:2781] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:18.099553 15559 raft_consensus.cc:482] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:18.099603 15559 raft_consensus.cc:3037] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:18.100677 15559 raft_consensus.cc:504] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.101023 15559 leader_election.cc:288] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [CANDIDATE]: Term 1 election: Requested vote from peers 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194:33307), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:18.101329 15261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e15df85b09045b187f19a534b7284c8" candidate_uuid: "940d425fb5ed45f48d81977dd94ac2c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08edb91831b34a088ad1f11ec9d99dc4"
I20260619 18:47:18.101447 15261 raft_consensus.cc:3037] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:18.101604 15400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e15df85b09045b187f19a534b7284c8" candidate_uuid: "940d425fb5ed45f48d81977dd94ac2c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b"
I20260619 18:47:18.101714 15400 raft_consensus.cc:3037] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:18.102517 15261 raft_consensus.cc:2454] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 940d425fb5ed45f48d81977dd94ac2c9 in term 1.
I20260619 18:47:18.102725 15400 raft_consensus.cc:2454] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 940d425fb5ed45f48d81977dd94ac2c9 in term 1.
I20260619 18:47:18.102770 15054 leader_election.cc:302] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08edb91831b34a088ad1f11ec9d99dc4, 940d425fb5ed45f48d81977dd94ac2c9; no voters:
I20260619 18:47:18.102927 15559 raft_consensus.cc:2781] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:18.103148 15559 raft_consensus.cc:686] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 LEADER]: Becoming Leader. State: Replica: 940d425fb5ed45f48d81977dd94ac2c9, State: Running, Role: LEADER
I20260619 18:47:18.103384 15559 consensus_queue.cc:237] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.105718 14964 catalog_manager.cc:5495] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193). New cstate: current_term: 1 leader_uuid: "940d425fb5ed45f48d81977dd94ac2c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } health_report { overall_health: UNKNOWN } } }
I20260619 18:47:18.152406 15568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:18.148682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46870 (local address 127.12.234.193:39741)
0619 18:47:18.148970 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:18.148979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:18.148991 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:47:18.149060 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:18.149062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:18.149166 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:18.149277 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:18.149510 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.150039 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.150916 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.151180 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.151267 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:18.151523 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:18.152002 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:18.152117 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:18.152185 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 18:47:18.152246 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":121,"threads_started":1}
I20260619 18:47:18.163055 15571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:18.159737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:33307 (local address 127.12.234.193:36977)
0619 18:47:18.160056 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:18.160069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:18.160184 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:18.160404 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:18.160407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:18.160414 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:18.160534 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:18.160538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.161266 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.161270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:18.161895 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:18.161903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.162350 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.162353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:18.162445 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:18.162488 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:18.162752 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:18.162758 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:47:18.162810 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 18:47:18.162888 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":104,"threads_started":1}
I20260619 18:47:18.163033 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:18.159850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:36977 (local address 127.12.234.194:33307)
0619 18:47:18.159983 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:18.159988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:18.160185 (+ 197us) server_negotiation.cc:407] Connection header received
0619 18:47:18.160274 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:18.160277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:18.160322 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:18.160380 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:18.160648 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.161160 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.162007 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.162248 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.162334 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:18.162606 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:18.162681 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:18.162737 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:18.162832 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 18:47:18.162888 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:47:18.169085 15261 raft_consensus.cc:1261] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Refusing update from remote peer 940d425fb5ed45f48d81977dd94ac2c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:18.169647 15565 consensus_queue.cc:1035] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:18.169756 15400 raft_consensus.cc:1261] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Refusing update from remote peer 940d425fb5ed45f48d81977dd94ac2c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:18.170068 15565 consensus_queue.cc:1035] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:18.188627 15568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:18.185541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:53873 (local address 127.12.234.193:39741)
0619 18:47:18.185641 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:18.185645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:18.185840 (+ 195us) server_negotiation.cc:407] Connection header received
0619 18:47:18.185959 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:18.185962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:18.186009 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:18.186072 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:18.186396 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.186910 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.187685 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.187898 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.187980 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:18.188194 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:18.188277 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:18.188343 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:18.188446 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 18:47:18.188494 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 18:47:18.188683 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:18.185501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:39741 (local address 127.12.234.194:53873)
0619 18:47:18.185758 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:18.185770 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:18.185866 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:18.186204 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:18.186207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:18.186216 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:18.186308 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:18.186312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.186988 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.186990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:18.187582 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:18.187589 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:18.187986 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:18.187989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:18.188075 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:18.188099 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:18.188358 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:18.188366 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 18:47:18.188438 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 18:47:18.188538 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":102,"threads_started":1}
I20260619 18:47:18.211669 14962 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:18.211831 14962 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:18.214152 14962 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc: Sending DeleteTablet for 3 replicas of tablet 3e15df85b09045b187f19a534b7284c8
I20260619 18:47:18.214847 15101 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e15df85b09045b187f19a534b7284c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:18 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:46862
I20260619 18:47:18.214833 15228 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e15df85b09045b187f19a534b7284c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:18 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:54132
I20260619 18:47:18.215023 15543 tablet_replica.cc:330] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: stopping tablet replica
I20260619 18:47:18.215023 15542 tablet_replica.cc:330] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: stopping tablet replica
I20260619 18:47:18.215131 15543 raft_consensus.cc:2229] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:18.215137 15542 raft_consensus.cc:2229] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:18.215240 15543 raft_consensus.cc:2258] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:18.215458 15542 raft_consensus.cc:2258] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:18.215667 15381 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e15df85b09045b187f19a534b7284c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:18 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:40068
I20260619 18:47:18.215934 15544 tablet_replica.cc:330] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: stopping tablet replica
I20260619 18:47:18.216054 15544 raft_consensus.cc:2229] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:18.216166 15544 raft_consensus.cc:2258] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:18.216572 15542 ts_tablet_manager.cc:1905] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:18.217037 15544 ts_tablet_manager.cc:1905] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:18.218426 14962 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:18.218940 14962 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:18.219187 15542 ts_tablet_manager.cc:1918] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:18.219259 15542 log.cc:1196] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/3e15df85b09045b187f19a534b7284c8
I20260619 18:47:18.219460 15544 ts_tablet_manager.cc:1918] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:18.219486 15542 ts_tablet_manager.cc:1939] T 3e15df85b09045b187f19a534b7284c8 P 940d425fb5ed45f48d81977dd94ac2c9: Deleting consensus metadata
I20260619 18:47:18.219520 15544 log.cc:1196] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/3e15df85b09045b187f19a534b7284c8
I20260619 18:47:18.219753 15544 ts_tablet_manager.cc:1939] T 3e15df85b09045b187f19a534b7284c8 P 0b8343ebfcfa45b08267cb624b6af83b: Deleting consensus metadata
I20260619 18:47:18.220227 14951 catalog_manager.cc:4841] TS 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741): tablet 3e15df85b09045b187f19a534b7284c8 (table test-table [id=22ec520903264571bc5b18c3b9f2bd3d]) successfully deleted
I20260619 18:47:18.220402 14948 catalog_manager.cc:4841] TS 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875): tablet 3e15df85b09045b187f19a534b7284c8 (table test-table [id=22ec520903264571bc5b18c3b9f2bd3d]) successfully deleted
I20260619 18:47:18.221045 15543 ts_tablet_manager.cc:1905] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:18.223559 15543 ts_tablet_manager.cc:1918] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:18.223650 15543 log.cc:1196] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/3e15df85b09045b187f19a534b7284c8
I20260619 18:47:18.223877 15543 ts_tablet_manager.cc:1939] T 3e15df85b09045b187f19a534b7284c8 P 08edb91831b34a088ad1f11ec9d99dc4: Deleting consensus metadata
I20260619 18:47:18.224522 14950 catalog_manager.cc:4841] TS 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194:33307): tablet 3e15df85b09045b187f19a534b7284c8 (table test-table [id=22ec520903264571bc5b18c3b9f2bd3d]) successfully deleted
I20260619 18:47:18.226426 15101 tablet_service.cc:1467] Processing CreateTablet for tablet 52e5116b2cd74e019902946f7dcc046f (DEFAULT_TABLE table=test-table [id=8cd34000e2984a9581b33757bfc099e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:18.226459 15228 tablet_service.cc:1467] Processing CreateTablet for tablet 52e5116b2cd74e019902946f7dcc046f (DEFAULT_TABLE table=test-table [id=8cd34000e2984a9581b33757bfc099e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:18.226682 15228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e5116b2cd74e019902946f7dcc046f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:18.226682 15101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e5116b2cd74e019902946f7dcc046f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:18.228967 15553 tablet_bootstrap.cc:492] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4: Bootstrap starting.
I20260619 18:47:18.229187 15381 tablet_service.cc:1467] Processing CreateTablet for tablet 52e5116b2cd74e019902946f7dcc046f (DEFAULT_TABLE table=test-table [id=8cd34000e2984a9581b33757bfc099e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:18.229421 15381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e5116b2cd74e019902946f7dcc046f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:18.229863 15553 tablet_bootstrap.cc:654] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:18.230418 15555 tablet_bootstrap.cc:492] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9: Bootstrap starting.
I20260619 18:47:18.231359 15555 tablet_bootstrap.cc:654] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:18.231803 15554 tablet_bootstrap.cc:492] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: Bootstrap starting.
I20260619 18:47:18.232286 15553 tablet_bootstrap.cc:492] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4: No bootstrap required, opened a new log
I20260619 18:47:18.232364 15553 ts_tablet_manager.cc:1397] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:18.232705 15554 tablet_bootstrap.cc:654] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:18.232766 15553 raft_consensus.cc:348] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.232877 15553 raft_consensus.cc:374] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:18.232906 15553 raft_consensus.cc:729] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08edb91831b34a088ad1f11ec9d99dc4, State: Initialized, Role: FOLLOWER
I20260619 18:47:18.233029 15553 consensus_queue.cc:260] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.233283 15553 ts_tablet_manager.cc:1428] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 18:47:18.234156 15555 tablet_bootstrap.cc:492] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9: No bootstrap required, opened a new log
I20260619 18:47:18.234232 15554 tablet_bootstrap.cc:492] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: No bootstrap required, opened a new log
I20260619 18:47:18.234282 15555 ts_tablet_manager.cc:1397] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 18:47:18.234316 15554 ts_tablet_manager.cc:1397] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:18.234717 15555 raft_consensus.cc:348] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.234834 15555 raft_consensus.cc:374] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:18.234748 15554 raft_consensus.cc:348] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.234863 15555 raft_consensus.cc:729] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940d425fb5ed45f48d81977dd94ac2c9, State: Initialized, Role: FOLLOWER
I20260619 18:47:18.234871 15554 raft_consensus.cc:374] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:18.234902 15554 raft_consensus.cc:729] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8343ebfcfa45b08267cb624b6af83b, State: Initialized, Role: FOLLOWER
I20260619 18:47:18.234973 15555 consensus_queue.cc:260] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.235019 15554 consensus_queue.cc:260] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.235226 15555 ts_tablet_manager.cc:1428] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 18:47:18.235280 15554 ts_tablet_manager.cc:1428] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:18.387740 15577 raft_consensus.cc:482] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:18.387907 15577 raft_consensus.cc:504] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.388324 15577 leader_election.cc:288] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194:33307), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:18.388779 15400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e5116b2cd74e019902946f7dcc046f" candidate_uuid: "940d425fb5ed45f48d81977dd94ac2c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b" is_pre_election: true
I20260619 18:47:18.388780 15261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e5116b2cd74e019902946f7dcc046f" candidate_uuid: "940d425fb5ed45f48d81977dd94ac2c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08edb91831b34a088ad1f11ec9d99dc4" is_pre_election: true
I20260619 18:47:18.388923 15261 raft_consensus.cc:2454] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 940d425fb5ed45f48d81977dd94ac2c9 in term 0.
I20260619 18:47:18.388931 15400 raft_consensus.cc:2454] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 940d425fb5ed45f48d81977dd94ac2c9 in term 0.
I20260619 18:47:18.389271 15054 leader_election.cc:302] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08edb91831b34a088ad1f11ec9d99dc4, 0b8343ebfcfa45b08267cb624b6af83b, 940d425fb5ed45f48d81977dd94ac2c9; no voters:
I20260619 18:47:18.389480 15577 raft_consensus.cc:2781] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:18.389552 15577 raft_consensus.cc:482] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:18.389598 15577 raft_consensus.cc:3037] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:18.390666 15577 raft_consensus.cc:504] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.391040 15577 leader_election.cc:288] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [CANDIDATE]: Term 1 election: Requested vote from peers 08edb91831b34a088ad1f11ec9d99dc4 (127.12.234.194:33307), 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875)
I20260619 18:47:18.391476 15261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e5116b2cd74e019902946f7dcc046f" candidate_uuid: "940d425fb5ed45f48d81977dd94ac2c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08edb91831b34a088ad1f11ec9d99dc4"
I20260619 18:47:18.391544 15400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e5116b2cd74e019902946f7dcc046f" candidate_uuid: "940d425fb5ed45f48d81977dd94ac2c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8343ebfcfa45b08267cb624b6af83b"
I20260619 18:47:18.391604 15261 raft_consensus.cc:3037] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:18.391675 15400 raft_consensus.cc:3037] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:18.392510 15261 raft_consensus.cc:2454] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 940d425fb5ed45f48d81977dd94ac2c9 in term 1.
I20260619 18:47:18.392627 15400 raft_consensus.cc:2454] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 940d425fb5ed45f48d81977dd94ac2c9 in term 1.
I20260619 18:47:18.392828 15054 leader_election.cc:302] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08edb91831b34a088ad1f11ec9d99dc4, 940d425fb5ed45f48d81977dd94ac2c9; no voters:
I20260619 18:47:18.392976 15577 raft_consensus.cc:2781] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:18.393074 15577 raft_consensus.cc:686] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [term 1 LEADER]: Becoming Leader. State: Replica: 940d425fb5ed45f48d81977dd94ac2c9, State: Running, Role: LEADER
I20260619 18:47:18.393209 15577 consensus_queue.cc:237] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } }
I20260619 18:47:18.394845 14962 catalog_manager.cc:5495] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193). New cstate: current_term: 1 leader_uuid: "940d425fb5ed45f48d81977dd94ac2c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "940d425fb5ed45f48d81977dd94ac2c9" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 } health_report { overall_health: UNKNOWN } } }
I20260619 18:47:18.435992 15400 raft_consensus.cc:1261] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Refusing update from remote peer 940d425fb5ed45f48d81977dd94ac2c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:18.436165 15261 raft_consensus.cc:1261] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Refusing update from remote peer 940d425fb5ed45f48d81977dd94ac2c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:18.436365 15565 consensus_queue.cc:1035] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b8343ebfcfa45b08267cb624b6af83b" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:18.436833 15565 consensus_queue.cc:1035] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08edb91831b34a088ad1f11ec9d99dc4" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 33307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 18:47:18.449141 15527 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 18:47:19.999699 14964 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:19.999845 14964 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:20.002311 14964 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 128c380803c446148152adaaa32c2bfc: Sending DeleteTablet for 3 replicas of tablet 52e5116b2cd74e019902946f7dcc046f
I20260619 18:47:20.002839 15101 tablet_service.cc:1514] Processing DeleteTablet for tablet 52e5116b2cd74e019902946f7dcc046f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:19 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:46862
I20260619 18:47:20.002885 15228 tablet_service.cc:1514] Processing DeleteTablet for tablet 52e5116b2cd74e019902946f7dcc046f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:19 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:54132
I20260619 18:47:20.003037 15381 tablet_service.cc:1514] Processing DeleteTablet for tablet 52e5116b2cd74e019902946f7dcc046f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:19 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:40068
I20260619 18:47:20.003196 15605 tablet_replica.cc:330] T 52e5116b2cd74e019902946f7dcc046f P 940d425fb5ed45f48d81977dd94ac2c9: stopping tablet replica
I20260619 18:47:20.003302 15606 tablet_replica.cc:330] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4: stopping tablet replica
I20260619 18:47:20.003346 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 15036
I20260619 18:47:20.003892 15606 raft_consensus.cc:2229] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:20.004014 15606 raft_consensus.cc:2258] T 52e5116b2cd74e019902946f7dcc046f P 08edb91831b34a088ad1f11ec9d99dc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:20.005116 15607 tablet_replica.cc:330] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: stopping tablet replica
I20260619 18:47:20.005211 15607 raft_consensus.cc:2229] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:20.005295 15607 raft_consensus.cc:2258] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:20.005856 15607 ts_tablet_manager.cc:1905] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:20.008397 15607 ts_tablet_manager.cc:1918] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 18:47:20.008453 15607 log.cc:1196] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/52e5116b2cd74e019902946f7dcc046f
I20260619 18:47:20.008639 15607 ts_tablet_manager.cc:1939] T 52e5116b2cd74e019902946f7dcc046f P 0b8343ebfcfa45b08267cb624b6af83b: Deleting consensus metadata
I20260619 18:47:20.009229 14948 catalog_manager.cc:4841] TS 0b8343ebfcfa45b08267cb624b6af83b (127.12.234.195:33875): tablet 52e5116b2cd74e019902946f7dcc046f (table test-table [id=8cd34000e2984a9581b33757bfc099e9]) successfully deleted
W20260619 18:47:20.010074 14951 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:39741 (error 108)
I20260619 18:47:20.010218 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 15176
I20260619 18:47:20.011998 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:20.011292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:39741 (local address 127.0.0.1:49070)
0619 18:47:20.011851 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:20.011916 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:39741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":77,"threads_started":1}
W20260619 18:47:20.012130 14951 catalog_manager.cc:4584] TS 940d425fb5ed45f48d81977dd94ac2c9 (127.12.234.193:39741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52e5116b2cd74e019902946f7dcc046f: Network error: Client connection negotiation failed: client connection to 127.12.234.193:39741: connect: Connection refused (error 111)
I20260619 18:47:20.016381 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 15316
I20260619 18:47:20.017256 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:20.016605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:33307 (local address 127.0.0.1:36684)
0619 18:47:20.017113 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:20.017171 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.194:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424}
I20260619 18:47:20.021618 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 14932
2026-06-19T18:47:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6297 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:20 dist-test-slave-fljv krb5kdc[15612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:20 dist-test-slave-fljv krb5kdc[15612](info): set up 2 sockets
Jun 19 18:47:20 dist-test-slave-fljv krb5kdc[15612](info): commencing operation
krb5kdc: starting...
W20260619 18:47:20.985841 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:21 dist-test-slave-fljv krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:21.104313 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:44561
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:44561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:21.213907 15625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:21.214236 15625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:21.214279 15625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:21.218442 15625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:21.218531 15625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:21.218556 15625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:21.218573 15625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:21.218592 15625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:21.224146 15625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:44561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:44561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:21.225288 15625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:21.226390 15625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:21.234856 15630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:21.235011 15625 server_base.cc:1034] running on GCE node
W20260619 18:47:21.234879 15631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:21.234879 15633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:21.235707 15625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:21.236788 15625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:21.237942 15625 hybrid_clock.cc:648] HybridClock initialized: now 1781894841237923 us; error 35 us; skew 500 ppm
Jun 19 18:47:21 dist-test-slave-fljv krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894841, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:21.245811 15625 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:21.246835 15625 webserver.cc:460] Webserver started at http://127.12.234.254:43895/ using document root <none> and password file <none>
I20260619 18:47:21.247444 15625 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:21.247516 15625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:21.247721 15625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:21.249843 15625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "5665add2d6e94501ab8c7eb3a6478643"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "6a96c9f7c0a35cc5ba70abeb49d47060"
server_key_iv: "eaa94f9141d23fff38dff8dfd5166b8d"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.250409 15625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "5665add2d6e94501ab8c7eb3a6478643"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "6a96c9f7c0a35cc5ba70abeb49d47060"
server_key_iv: "eaa94f9141d23fff38dff8dfd5166b8d"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.254135 15625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260619 18:47:21.256532 15640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:21.257791 15625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:21.257923 15625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "5665add2d6e94501ab8c7eb3a6478643"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "6a96c9f7c0a35cc5ba70abeb49d47060"
server_key_iv: "eaa94f9141d23fff38dff8dfd5166b8d"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.258024 15625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:21.275743 15625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:21.278945 15625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:21.279134 15625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:21.285938 15625 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:44561
I20260619 18:47:21.285954 15692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:44561 every 8 connection(s)
I20260619 18:47:21.286978 15625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:21.289836 15693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:21.290623 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 15625
I20260619 18:47:21.290747 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:21.291069 13227 external_mini_cluster.cc:1428] Setting key 40bce3ddea8976ef905a81c163fe5a4a
I20260619 18:47:21.297848 15693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643: Bootstrap starting.
Jun 19 18:47:21 dist-test-slave-fljv krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:21.300649 15693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:21.301517 15693 log.cc:824] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:21.303988 15693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643: No bootstrap required, opened a new log
I20260619 18:47:21.306223 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:21.292530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47262 (local address 127.12.234.254:44561)
0619 18:47:21.292951 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:21.292972 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:21.293019 (+ 47us) server_negotiation.cc:407] Connection header received
0619 18:47:21.293742 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:21.293767 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:21.294238 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:21.294649 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:21.295341 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.296832 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.297740 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.298461 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.298647 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:21.301079 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:21.301087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:21.301104 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:21.301139 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:21.303299 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:21.303729 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:21.303735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:21.303741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:21.303876 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:21.304232 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:21.304235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:21.304236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:21.304659 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:21.304831 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:21.305174 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 18:47:21.305453 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":109,"threads_started":1}
I20260619 18:47:21.307673 15693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5665add2d6e94501ab8c7eb3a6478643" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44561 } }
I20260619 18:47:21.307897 15693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:21.307952 15693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5665add2d6e94501ab8c7eb3a6478643, State: Initialized, Role: FOLLOWER
I20260619 18:47:21.308378 15693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5665add2d6e94501ab8c7eb3a6478643" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44561 } }
I20260619 18:47:21.308532 15693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:21.308601 15693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:21.308694 15693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:21.309554 15693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5665add2d6e94501ab8c7eb3a6478643" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44561 } }
I20260619 18:47:21.309877 15693 leader_election.cc:302] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5665add2d6e94501ab8c7eb3a6478643; no voters:
I20260619 18:47:21.310199 15693 leader_election.cc:288] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:21.310324 15698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:21.310514 15698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [term 1 LEADER]: Becoming Leader. State: Replica: 5665add2d6e94501ab8c7eb3a6478643, State: Running, Role: LEADER
I20260619 18:47:21.310815 15698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5665add2d6e94501ab8c7eb3a6478643" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44561 } }
I20260619 18:47:21.311369 15693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:21.312332 15700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5665add2d6e94501ab8c7eb3a6478643. Latest consensus state: current_term: 1 leader_uuid: "5665add2d6e94501ab8c7eb3a6478643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5665add2d6e94501ab8c7eb3a6478643" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44561 } } }
I20260619 18:47:21.312469 15700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:21.312394 15699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5665add2d6e94501ab8c7eb3a6478643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5665add2d6e94501ab8c7eb3a6478643" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44561 } } }
I20260619 18:47:21.312773 15699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:21.312830 15707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:21.316215 15707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:21.323144 15707 catalog_manager.cc:1347] Generated new cluster ID: 7d7c4cd29e654b6b94f86ce4f6ba30d9
I20260619 18:47:21.323232 15707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:21.337802 15707 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:21.339792 15707 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:21.363893 15707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5665add2d6e94501ab8c7eb3a6478643: Generated new TSK 0
I20260619 18:47:21.364691 15707 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:21.401671 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44561
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:21.522526 15719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:21.522825 15719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:21.522873 15719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:21.527130 15719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:21.527220 15719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:21.527287 15719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:21.533217 15719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:21.534425 15719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:21.535485 15719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:21.545012 15727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:21.545094 15725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:21.545012 15724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:21.545202 15719 server_base.cc:1034] running on GCE node
I20260619 18:47:21.545670 15719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:21.546347 15719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:21.547516 15719 hybrid_clock.cc:648] HybridClock initialized: now 1781894841547502 us; error 26 us; skew 500 ppm
Jun 19 18:47:21 dist-test-slave-fljv krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894841, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:21.550439 15719 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:21.551304 15719 webserver.cc:460] Webserver started at http://127.12.234.193:39283/ using document root <none> and password file <none>
I20260619 18:47:21.551812 15719 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:21.551869 15719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:21.552035 15719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:21.554004 15719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "bebbcc86fb974b43851c34c3d093f11e"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "8afc197903799bff973a8c0426c3e503"
server_key_iv: "4f3ec5669d33f26e2561f3cb097eef6b"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.554565 15719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "bebbcc86fb974b43851c34c3d093f11e"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "8afc197903799bff973a8c0426c3e503"
server_key_iv: "4f3ec5669d33f26e2561f3cb097eef6b"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.558419 15719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:47:21.561048 15734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:21.562614 15719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:21.562733 15719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "bebbcc86fb974b43851c34c3d093f11e"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "8afc197903799bff973a8c0426c3e503"
server_key_iv: "4f3ec5669d33f26e2561f3cb097eef6b"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.562841 15719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:21.585006 15719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:21.588289 15719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:21.588472 15719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:21.589063 15719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:21.589880 15719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:21.589927 15719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:21.589979 15719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:21.590005 15719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:21.598124 15719 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:33643
I20260619 18:47:21.598160 15847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:33643 every 8 connection(s)
I20260619 18:47:21.599071 15719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
Jun 19 18:47:21 dist-test-slave-fljv krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894841, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:21.608917 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 15719
I20260619 18:47:21.609016 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:21.609268 13227 external_mini_cluster.cc:1428] Setting key a0d633532953b1d5bd10a62e0ce9cf29
I20260619 18:47:21.613413 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:21.600815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:51861 (local address 127.12.234.254:44561)
0619 18:47:21.600938 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:21.600941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:21.602252 (+ 1311us) server_negotiation.cc:407] Connection header received
0619 18:47:21.603208 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:21.603211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:21.603272 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:21.603346 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:21.604640 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.605186 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.606480 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.606757 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.606851 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:21.609440 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:21.609444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:21.609447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:21.609479 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:21.611088 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:21.611568 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:21.611571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:21.611573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:21.611662 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:21.612073 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:21.612077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:21.612078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:21.612287 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:21.612381 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:21.613178 (+ 797us) server_negotiation.cc:299] Negotiation successful
0619 18:47:21.613290 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:47:21.614593 15850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:21.601164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44561 (local address 127.12.234.193:51861)
0619 18:47:21.602057 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:21.602104 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:21.602990 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:21.603523 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:21.603529 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:21.603915 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:21.604473 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:21.604488 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.605318 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.605321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:21.606359 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:21.606368 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.606905 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.606910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:21.607109 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:21.607672 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:21.607688 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:21.609255 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:21.611194 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:21.611199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:21.611213 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:21.611477 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:21.611737 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:21.611739 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:21.611740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:21.611908 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:21.612413 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:21.612421 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:21.612767 (+ 346us) client_negotiation.cc:769] Sending connection context
0619 18:47:21.613128 (+ 361us) client_negotiation.cc:240] Negotiation successful
0619 18:47:21.613509 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":511,"threads_started":1}
I20260619 18:47:21.615759 15848 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44561
I20260619 18:47:21.615995 15848 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:21.616489 15848 heartbeater.cc:511] Master 127.12.234.254:44561 requested a full tablet report, sending...
I20260619 18:47:21.617910 15657 ts_manager.cc:194] Registered new tserver with Master: bebbcc86fb974b43851c34c3d093f11e (127.12.234.193:33643)
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:21.619081 15657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:51861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:21.638144 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44561
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:21.747025 15853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:21.747296 15853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:21.747340 15853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:21.751427 15853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:21.751493 15853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:21.751559 15853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:21.757061 15853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:21.758095 15853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:21.759133 15853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:21.768653 15859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:21.768653 15861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:21.768709 15853 server_base.cc:1034] running on GCE node
W20260619 18:47:21.768666 15858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:21.769340 15853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:21.769982 15853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:21.771145 15853 hybrid_clock.cc:648] HybridClock initialized: now 1781894841771136 us; error 62 us; skew 500 ppm
Jun 19 18:47:21 dist-test-slave-fljv krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894841, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:21.773758 15853 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:21.774693 15853 webserver.cc:460] Webserver started at http://127.12.234.194:36977/ using document root <none> and password file <none>
I20260619 18:47:21.775229 15853 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:21.775291 15853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:21.775486 15853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:21.777676 15853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "7e3bbe72c433412ca9b9f2cb8d3f3deb"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "cfc1e91fb4630c17bdba79a864339e78"
server_key_iv: "e6adb12f4ed5e07319deb7820a077402"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.778271 15853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "7e3bbe72c433412ca9b9f2cb8d3f3deb"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "cfc1e91fb4630c17bdba79a864339e78"
server_key_iv: "e6adb12f4ed5e07319deb7820a077402"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.781896 15853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:21.784439 15868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:21.785665 15853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:21.785774 15853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "7e3bbe72c433412ca9b9f2cb8d3f3deb"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "cfc1e91fb4630c17bdba79a864339e78"
server_key_iv: "e6adb12f4ed5e07319deb7820a077402"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.785876 15853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:21.806797 15853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:21.809931 15853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:21.810114 15853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:21.810734 15853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:21.811585 15853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:21.811635 15853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:21.811688 15853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:21.811712 15853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:21.820381 15853 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:40145
I20260619 18:47:21.820410 15981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:40145 every 8 connection(s)
I20260619 18:47:21.821296 15853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:21.824497 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 15853
I20260619 18:47:21.824595 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:21.824870 13227 external_mini_cluster.cc:1428] Setting key e5ebc3359e49263d979053824e19b452
Jun 19 18:47:21 dist-test-slave-fljv krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894841, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:47:21.835879 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:21.823133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:33399 (local address 127.12.234.254:44561)
0619 18:47:21.823248 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:21.823251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:21.824296 (+ 1045us) server_negotiation.cc:407] Connection header received
0619 18:47:21.825408 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:21.825412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:21.825469 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:21.825558 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:21.827490 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.828089 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.829512 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.829852 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.829955 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:21.832479 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:21.832482 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:21.832484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:21.832509 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:21.833829 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:21.834390 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:21.834392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:21.834394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:21.834467 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:21.834786 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:21.834789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:21.834791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:21.834993 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:21.835092 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:21.835584 (+ 492us) server_negotiation.cc:299] Negotiation successful
0619 18:47:21.835707 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:47:21.836874 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:21.823472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44561 (local address 127.12.234.194:33399)
0619 18:47:21.824140 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:21.824183 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:21.825186 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:21.826140 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:21.826147 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:21.826656 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:21.827349 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:21.827368 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.828208 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.828212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:21.829396 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:21.829406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:21.829982 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:21.829985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:21.830122 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:21.830692 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:21.830709 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:21.832314 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:21.833939 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:21.833945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:21.833962 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:21.834294 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:21.834544 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:21.834546 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:21.834549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:21.834694 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:21.835085 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:21.835092 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:21.835349 (+ 257us) client_negotiation.cc:769] Sending connection context
0619 18:47:21.835514 (+ 165us) client_negotiation.cc:240] Negotiation successful
0619 18:47:21.835872 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":109,"threads_started":1}
I20260619 18:47:21.838306 15982 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44561
I20260619 18:47:21.838608 15982 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:21.839215 15982 heartbeater.cc:511] Master 127.12.234.254:44561 requested a full tablet report, sending...
I20260619 18:47:21.840366 15657 ts_manager.cc:194] Registered new tserver with Master: 7e3bbe72c433412ca9b9f2cb8d3f3deb (127.12.234.194:40145)
I20260619 18:47:21.840927 15657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:33399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:21.853789 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44561
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:21.965436 15987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:21.965716 15987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:21.965761 15987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:21.970031 15987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:21.970104 15987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:21.970167 15987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:21.975787 15987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:21.976889 15987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:21.977924 15987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:21.987716 15995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:21.987720 15992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:21.987826 15987 server_base.cc:1034] running on GCE node
W20260619 18:47:21.987712 15993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:21.988485 15987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:21.989220 15987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:21.990388 15987 hybrid_clock.cc:648] HybridClock initialized: now 1781894841990364 us; error 34 us; skew 500 ppm
Jun 19 18:47:21 dist-test-slave-fljv krb5kdc[15612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894841, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:21.993209 15987 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:21.994122 15987 webserver.cc:460] Webserver started at http://127.12.234.195:33605/ using document root <none> and password file <none>
I20260619 18:47:21.994694 15987 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:21.994764 15987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:21.994978 15987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:21.997128 15987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "52422b1fbf544182961985d860950b8d"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "bafe4acba4c944930f84acbb32e82288"
server_key_iv: "60b7588c5a79341b2615ebea2fc4dcbd"
server_key_version: "encryptionkey@0"
I20260619 18:47:21.997714 15987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "52422b1fbf544182961985d860950b8d"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "bafe4acba4c944930f84acbb32e82288"
server_key_iv: "60b7588c5a79341b2615ebea2fc4dcbd"
server_key_version: "encryptionkey@0"
I20260619 18:47:22.001710 15987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:22.004230 16002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:22.005481 15987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:22.005599 15987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "52422b1fbf544182961985d860950b8d"
format_stamp: "Formatted at 2026-06-19 18:47:21 on dist-test-slave-fljv"
server_key: "bafe4acba4c944930f84acbb32e82288"
server_key_iv: "60b7588c5a79341b2615ebea2fc4dcbd"
server_key_version: "encryptionkey@0"
I20260619 18:47:22.005700 15987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:22.037153 15987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:22.040328 15987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:22.040504 15987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:22.041097 15987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:22.041940 15987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:22.041993 15987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:22.042047 15987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:22.042071 15987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:22.050555 15987 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:36805
I20260619 18:47:22.050607 16115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:36805 every 8 connection(s)
I20260619 18:47:22.051432 15987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
Jun 19 18:47:22 dist-test-slave-fljv krb5kdc[15612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894841, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:22.060566 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 15987
I20260619 18:47:22.060686 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:22.060962 13227 external_mini_cluster.cc:1428] Setting key 90d460e18ee36eb925ae869118c208a2
I20260619 18:47:22.064602 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:22.053202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:37673 (local address 127.12.234.254:44561)
0619 18:47:22.053393 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:22.053398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:22.054503 (+ 1105us) server_negotiation.cc:407] Connection header received
0619 18:47:22.055419 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:22.055422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:22.055469 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:22.055559 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:22.056670 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:22.057189 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:22.058262 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:22.058477 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:22.058546 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:22.060902 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:22.060905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:22.060907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:22.060931 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:22.062524 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:22.063136 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:22.063139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:22.063140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:22.063218 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:22.063526 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:22.063529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:22.063530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:22.063730 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:22.063816 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:22.064377 (+ 561us) server_negotiation.cc:299] Negotiation successful
0619 18:47:22.064481 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 18:47:22.065433 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:22.053668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44561 (local address 127.12.234.195:37673)
0619 18:47:22.054331 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:22.054398 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:22.055228 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:22.055717 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:22.055723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:22.056069 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:22.056556 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:22.056569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:22.057277 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:22.057279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:22.058154 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:22.058162 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:22.058603 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:22.058606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:22.058745 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:22.059243 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:22.059259 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:22.060721 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:22.062625 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:22.062631 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:22.062649 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:22.063013 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:22.063282 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:22.063284 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:22.063285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:22.063419 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:22.063851 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:22.063857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:22.064126 (+ 269us) client_negotiation.cc:769] Sending connection context
0619 18:47:22.064300 (+ 174us) client_negotiation.cc:240] Negotiation successful
0619 18:47:22.064589 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":114,"threads_started":1}
I20260619 18:47:22.066563 16116 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44561
I20260619 18:47:22.066829 16116 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:22.067320 16116 heartbeater.cc:511] Master 127.12.234.254:44561 requested a full tablet report, sending...
I20260619 18:47:22.068359 15657 ts_manager.cc:194] Registered new tserver with Master: 52422b1fbf544182961985d860950b8d (127.12.234.195:36805)
I20260619 18:47:22.068897 15657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:37673
I20260619 18:47:22.074966 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 18:47:22.087908 15696 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:22.086495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.12.234.254:44561)
0619 18:47:22.086609 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:22.086612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:22.086854 (+ 242us) server_negotiation.cc:407] Connection header received
0619 18:47:22.086931 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:22.086933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:22.086978 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:22.087057 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:22.087766 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47268: BlockingRecv error: recv got EOF from 127.0.0.1:47268 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 18:47:22.089038 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 15719
I20260619 18:47:22.093811 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 15853
I20260619 18:47:22.098472 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 15987
I20260619 18:47:22.102749 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 15625
2026-06-19T18:47:22Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2079 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:22 dist-test-slave-fljv krb5kdc[16128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:22 dist-test-slave-fljv krb5kdc[16128](info): set up 2 sockets
Jun 19 18:47:22 dist-test-slave-fljv krb5kdc[16128](info): commencing operation
krb5kdc: starting...
W20260619 18:47:23.054100 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:23 dist-test-slave-fljv krb5kdc[16128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:23.174667 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40335
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:40335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:23.286155 16141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:23.286473 16141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:23.286533 16141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:23.290572 16141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:23.290640 16141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:23.290666 16141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:23.290685 16141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:23.290705 16141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:23.296142 16141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:40335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:23.297282 16141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:23.298650 16141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:23.307744 16146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:23.307722 16149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:23.307729 16147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:23.307912 16141 server_base.cc:1034] running on GCE node
I20260619 18:47:23.308602 16141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:23.309752 16141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:23.310906 16141 hybrid_clock.cc:648] HybridClock initialized: now 1781894843310888 us; error 37 us; skew 500 ppm
Jun 19 18:47:23 dist-test-slave-fljv krb5kdc[16128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894843, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:23.313648 16141 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:23.314625 16141 webserver.cc:460] Webserver started at http://127.12.234.254:46683/ using document root <none> and password file <none>
I20260619 18:47:23.315160 16141 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:23.315218 16141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:23.315385 16141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:23.317416 16141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "5cc2e8446b504312a9e59e27fab54b88"
format_stamp: "Formatted at 2026-06-19 18:47:23 on dist-test-slave-fljv"
server_key: "432fb10e21d2de5c46a5fada0725f091"
server_key_iv: "b57bc9d1e36fd3c8b001889ae8fda52f"
server_key_version: "encryptionkey@0"
I20260619 18:47:23.317948 16141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "5cc2e8446b504312a9e59e27fab54b88"
format_stamp: "Formatted at 2026-06-19 18:47:23 on dist-test-slave-fljv"
server_key: "432fb10e21d2de5c46a5fada0725f091"
server_key_iv: "b57bc9d1e36fd3c8b001889ae8fda52f"
server_key_version: "encryptionkey@0"
I20260619 18:47:23.321904 16141 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:47:23.324687 16156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:23.326131 16141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 18:47:23.326294 16141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "5cc2e8446b504312a9e59e27fab54b88"
format_stamp: "Formatted at 2026-06-19 18:47:23 on dist-test-slave-fljv"
server_key: "432fb10e21d2de5c46a5fada0725f091"
server_key_iv: "b57bc9d1e36fd3c8b001889ae8fda52f"
server_key_version: "encryptionkey@0"
I20260619 18:47:23.326406 16141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:23.337932 16141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:23.341320 16141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:23.341480 16141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:23.348421 16141 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:40335
I20260619 18:47:23.348740 16208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:40335 every 8 connection(s)
I20260619 18:47:23.349421 16141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:23.350775 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 16141
I20260619 18:47:23.350943 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:23.351235 13227 external_mini_cluster.cc:1428] Setting key 69059b240bf8f4766c8fd0f02d0fdabb
I20260619 18:47:23.353012 16209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:23.359673 16209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88: Bootstrap starting.
Jun 19 18:47:23 dist-test-slave-fljv krb5kdc[16128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:23.362140 16209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:23.362871 16209 log.cc:824] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:23.365125 16209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88: No bootstrap required, opened a new log
I20260619 18:47:23.365824 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:23.352575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.12.234.254:40335)
0619 18:47:23.352991 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:23.353007 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:23.353043 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:47:23.353723 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:23.353741 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:23.354091 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:23.354422 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:23.355363 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:23.356434 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:23.357612 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:23.358140 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:23.358330 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:23.361135 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:23.361141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:23.361155 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:23.361186 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:23.362863 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:23.363353 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:23.363360 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:23.363365 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:23.363539 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:23.363848 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:23.363851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:23.363852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:23.364270 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:23.364406 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:23.364734 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 18:47:23.365019 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":114,"threads_started":1}
I20260619 18:47:23.368633 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:47:23.368809 16209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc2e8446b504312a9e59e27fab54b88" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40335 } }
I20260619 18:47:23.369103 16209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:23.369176 16209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc2e8446b504312a9e59e27fab54b88, State: Initialized, Role: FOLLOWER
I20260619 18:47:23.369666 16209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc2e8446b504312a9e59e27fab54b88" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40335 } }
I20260619 18:47:23.369825 16209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:23.369887 16209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:23.369979 16209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:23.371145 16209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc2e8446b504312a9e59e27fab54b88" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40335 } }
I20260619 18:47:23.371479 16209 leader_election.cc:302] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cc2e8446b504312a9e59e27fab54b88; no voters:
I20260619 18:47:23.371771 16209 leader_election.cc:288] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:23.371866 16215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:23.372040 16215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [term 1 LEADER]: Becoming Leader. State: Replica: 5cc2e8446b504312a9e59e27fab54b88, State: Running, Role: LEADER
I20260619 18:47:23.372407 16215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc2e8446b504312a9e59e27fab54b88" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40335 } }
I20260619 18:47:23.373116 16209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:23.373626 16215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cc2e8446b504312a9e59e27fab54b88. Latest consensus state: current_term: 1 leader_uuid: "5cc2e8446b504312a9e59e27fab54b88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc2e8446b504312a9e59e27fab54b88" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40335 } } }
I20260619 18:47:23.373811 16215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:23.373992 16216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cc2e8446b504312a9e59e27fab54b88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc2e8446b504312a9e59e27fab54b88" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40335 } } }
I20260619 18:47:23.374156 16216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:23.374256 16221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:23.377703 16221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:23.383571 16221 catalog_manager.cc:1347] Generated new cluster ID: 3e1407c3a3b4417f881abeee70ec136a
I20260619 18:47:23.383651 16221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:23.388768 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:23.384125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.12.234.254:40335)
0619 18:47:23.384253 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:23.384258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:23.384511 (+ 253us) server_negotiation.cc:407] Connection header received
0619 18:47:23.384639 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:23.384642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:23.384683 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:23.384747 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:23.385282 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:23.385833 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:23.386730 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:23.386974 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:23.387064 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:23.387866 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:23.387869 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:23.387871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:23.388322 (+ 451us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:23.388350 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:23.388367 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:23.388445 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:23.388544 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 18:47:23.388606 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":46}
I20260619 18:47:23.399813 16221 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:23.400802 16221 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:23.409081 16221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cc2e8446b504312a9e59e27fab54b88: Generated new TSK 0
I20260619 18:47:23.410027 16221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:47:23.421810 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 16141
2026-06-19T18:47:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1318 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:23 dist-test-slave-fljv krb5kdc[16242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:23 dist-test-slave-fljv krb5kdc[16242](info): set up 2 sockets
Jun 19 18:47:23 dist-test-slave-fljv krb5kdc[16242](info): commencing operation
krb5kdc: starting...
W20260619 18:47:24.385895 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:24 dist-test-slave-fljv krb5kdc[16242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:24.507298 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35969
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:33217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:35969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:24.617950 16255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:24.618288 16255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:24.618348 16255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:24.622459 16255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:24.622534 16255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:24.622560 16255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:24.622581 16255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:24.622601 16255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:24.628082 16255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:33217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:35969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:24.629230 16255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:24.630291 16255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:24.638993 16261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:24.639000 16260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:24.639106 16255 server_base.cc:1034] running on GCE node
W20260619 18:47:24.638993 16263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:24.639837 16255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:24.641129 16255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:24.642325 16255 hybrid_clock.cc:648] HybridClock initialized: now 1781894844642299 us; error 43 us; skew 500 ppm
Jun 19 18:47:24 dist-test-slave-fljv krb5kdc[16242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894844, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:24.645357 16255 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:24.646351 16255 webserver.cc:460] Webserver started at http://127.12.234.254:38051/ using document root <none> and password file <none>
I20260619 18:47:24.646903 16255 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:24.646960 16255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:24.647138 16255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:24.649165 16255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "9e4b7232af3345babc5c77b4f456dbd0"
format_stamp: "Formatted at 2026-06-19 18:47:24 on dist-test-slave-fljv"
server_key: "9ff0514546743d1d2dea901b4fb3ef84"
server_key_iv: "399a208cd7985a61ec58542d3df327b9"
server_key_version: "encryptionkey@0"
I20260619 18:47:24.649665 16255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "9e4b7232af3345babc5c77b4f456dbd0"
format_stamp: "Formatted at 2026-06-19 18:47:24 on dist-test-slave-fljv"
server_key: "9ff0514546743d1d2dea901b4fb3ef84"
server_key_iv: "399a208cd7985a61ec58542d3df327b9"
server_key_version: "encryptionkey@0"
I20260619 18:47:24.653367 16255 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260619 18:47:24.655797 16270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:24.657104 16255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 18:47:24.657222 16255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "9e4b7232af3345babc5c77b4f456dbd0"
format_stamp: "Formatted at 2026-06-19 18:47:24 on dist-test-slave-fljv"
server_key: "9ff0514546743d1d2dea901b4fb3ef84"
server_key_iv: "399a208cd7985a61ec58542d3df327b9"
server_key_version: "encryptionkey@0"
I20260619 18:47:24.657332 16255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:24.671849 16255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:24.675338 16255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:24.675535 16255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:24.682580 16255 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:35969
I20260619 18:47:24.682833 16322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:35969 every 8 connection(s)
I20260619 18:47:24.683606 16255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:24.683796 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 16255
I20260619 18:47:24.683914 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:24.684185 13227 external_mini_cluster.cc:1428] Setting key b5da7b6f6c5e173707c0ba316599c5ae
I20260619 18:47:24.686818 16323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:24.693550 16323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0: Bootstrap starting.
Jun 19 18:47:24 dist-test-slave-fljv krb5kdc[16242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:24.696249 16323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:24.697150 16323 log.cc:824] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:24.699448 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:24.685668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.12.234.254:35969)
0619 18:47:24.686057 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:24.686072 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:24.686107 (+ 35us) server_negotiation.cc:407] Connection header received
0619 18:47:24.686789 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:24.686807 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:24.687175 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:24.687458 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:24.688448 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:24.689483 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:24.690741 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:24.691239 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:24.691368 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:24.694456 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:24.694463 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:24.694478 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:24.694508 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:24.696240 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:24.696808 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:24.696813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:24.696817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:24.696934 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:24.697286 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:24.697290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:24.697292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:24.697755 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:24.697935 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:24.698347 (+ 412us) server_negotiation.cc:299] Negotiation successful
0619 18:47:24.698660 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":108,"threads_started":1}
I20260619 18:47:24.699451 16323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0: No bootstrap required, opened a new log
I20260619 18:47:24.703321 16323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e4b7232af3345babc5c77b4f456dbd0" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35969 } }
I20260619 18:47:24.703543 16323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:24.703609 16323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4b7232af3345babc5c77b4f456dbd0, State: Initialized, Role: FOLLOWER
I20260619 18:47:24.704025 16323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e4b7232af3345babc5c77b4f456dbd0" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35969 } }
I20260619 18:47:24.704155 16323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:24.704221 16323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:24.704341 16323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:24.705307 16323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e4b7232af3345babc5c77b4f456dbd0" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35969 } }
I20260619 18:47:24.705636 16323 leader_election.cc:302] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e4b7232af3345babc5c77b4f456dbd0; no voters:
I20260619 18:47:24.705952 16323 leader_election.cc:288] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:24.706065 16328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:24.706327 16328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [term 1 LEADER]: Becoming Leader. State: Replica: 9e4b7232af3345babc5c77b4f456dbd0, State: Running, Role: LEADER
I20260619 18:47:24.706638 16328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e4b7232af3345babc5c77b4f456dbd0" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35969 } }
I20260619 18:47:24.707149 16323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:24.708175 16329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e4b7232af3345babc5c77b4f456dbd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e4b7232af3345babc5c77b4f456dbd0" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35969 } } }
I20260619 18:47:24.708212 16330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e4b7232af3345babc5c77b4f456dbd0. Latest consensus state: current_term: 1 leader_uuid: "9e4b7232af3345babc5c77b4f456dbd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e4b7232af3345babc5c77b4f456dbd0" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35969 } } }
I20260619 18:47:24.708287 16329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:24.708309 16330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:24.709404 16337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:24.712061 16337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:24.717639 16337 catalog_manager.cc:1347] Generated new cluster ID: def5bb6a173c4405b1b3a9a6a483b58d
I20260619 18:47:24.717710 16337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:24.731452 16337 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:24.732807 16337 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:24.743556 16337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e4b7232af3345babc5c77b4f456dbd0: Generated new TSK 0
I20260619 18:47:24.744405 16337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:24.775300 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35969
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:33217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:24.886404 16349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:24.886698 16349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:24.886761 16349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:24.890820 16349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:24.890877 16349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:24.890959 16349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:24.896509 16349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:33217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:24.897605 16349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:24.898694 16349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:24.908198 16354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:24.908247 16349 server_base.cc:1034] running on GCE node
W20260619 18:47:24.908102 16357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:24.908188 16355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:24.908811 16349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:24.909461 16349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:24.910615 16349 hybrid_clock.cc:648] HybridClock initialized: now 1781894844910588 us; error 38 us; skew 500 ppm
Jun 19 18:47:24 dist-test-slave-fljv krb5kdc[16242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894844, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:24.913203 16349 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:24.914122 16349 webserver.cc:460] Webserver started at http://127.12.234.193:34639/ using document root <none> and password file <none>
I20260619 18:47:24.914682 16349 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:24.914747 16349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:24.914942 16349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:24.916937 16349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "3b3f04c08f1d467e903350d950f29d1f"
format_stamp: "Formatted at 2026-06-19 18:47:24 on dist-test-slave-fljv"
server_key: "b719631e6702b188937cafd4d57aa28d"
server_key_iv: "4eea7f4329400cd9ff73b0ac62474a5d"
server_key_version: "encryptionkey@0"
I20260619 18:47:24.917472 16349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "3b3f04c08f1d467e903350d950f29d1f"
format_stamp: "Formatted at 2026-06-19 18:47:24 on dist-test-slave-fljv"
server_key: "b719631e6702b188937cafd4d57aa28d"
server_key_iv: "4eea7f4329400cd9ff73b0ac62474a5d"
server_key_version: "encryptionkey@0"
I20260619 18:47:24.921424 16349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:24.923976 16364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:24.925328 16349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:24.925441 16349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "3b3f04c08f1d467e903350d950f29d1f"
format_stamp: "Formatted at 2026-06-19 18:47:24 on dist-test-slave-fljv"
server_key: "b719631e6702b188937cafd4d57aa28d"
server_key_iv: "4eea7f4329400cd9ff73b0ac62474a5d"
server_key_version: "encryptionkey@0"
I20260619 18:47:24.925544 16349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:24.938027 16349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:24.941049 16349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:24.941220 16349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:24.941826 16349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:24.942687 16349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:24.942754 16349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:24.942827 16349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:24.942852 16349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:24.951100 16349 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:42831
I20260619 18:47:24.951138 16477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:42831 every 8 connection(s)
I20260619 18:47:24.952034 16349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
Jun 19 18:47:24 dist-test-slave-fljv krb5kdc[16242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894844, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:24.961961 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 16349
I20260619 18:47:24.962110 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:24.962410 13227 external_mini_cluster.cc:1428] Setting key 9d3349344d289ba2b95685feff5088a7
I20260619 18:47:24.966014 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:24.953882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:41693 (local address 127.12.234.254:35969)
0619 18:47:24.954049 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:24.954053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:24.955062 (+ 1009us) server_negotiation.cc:407] Connection header received
0619 18:47:24.956358 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:24.956361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:24.956409 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:24.956482 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:24.957733 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:24.958322 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:24.959537 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:24.959770 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:24.959851 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:24.962490 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:24.962494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:24.962497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:24.962528 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:24.964019 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:24.964495 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:24.964499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:24.964501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:24.964596 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:24.964904 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:24.964906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:24.964908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:24.965114 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:24.965200 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:24.965719 (+ 519us) server_negotiation.cc:299] Negotiation successful
0619 18:47:24.965854 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:47:24.966977 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:24.954407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35969 (local address 127.12.234.193:41693)
0619 18:47:24.954882 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:24.954926 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:24.956164 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:24.956665 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:24.956674 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:24.957078 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:24.957574 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:24.957589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:24.958446 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:24.958449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:24.959421 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:24.959430 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:24.959891 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:24.959893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:24.960069 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:24.960647 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:24.960667 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:24.962306 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:24.964121 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:24.964125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:24.964139 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:24.964402 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:24.964685 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:24.964687 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:24.964688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:24.964832 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:24.965214 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:24.965219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:24.965475 (+ 256us) client_negotiation.cc:769] Sending connection context
0619 18:47:24.965650 (+ 175us) client_negotiation.cc:240] Negotiation successful
0619 18:47:24.965918 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":109,"threads_started":1}
I20260619 18:47:24.968369 16478 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35969
I20260619 18:47:24.968622 16478 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:24.969102 16478 heartbeater.cc:511] Master 127.12.234.254:35969 requested a full tablet report, sending...
I20260619 18:47:24.970794 16287 ts_manager.cc:194] Registered new tserver with Master: 3b3f04c08f1d467e903350d950f29d1f (127.12.234.193:42831)
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:24.972358 16287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:41693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:24.989483 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35969
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:33217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:25.100260 16483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:25.100528 16483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:25.100569 16483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:25.104636 16483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:25.104691 16483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:25.104758 16483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:25.110296 16483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:33217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:25.111370 16483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:25.112385 16483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:25.121937 16488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:25.121953 16491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:25.122304 16483 server_base.cc:1034] running on GCE node
W20260619 18:47:25.121995 16489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:25.122694 16483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:25.123284 16483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:25.124445 16483 hybrid_clock.cc:648] HybridClock initialized: now 1781894845124424 us; error 31 us; skew 500 ppm
Jun 19 18:47:25 dist-test-slave-fljv krb5kdc[16242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894845, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:25.130541 16483 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:25.131644 16483 webserver.cc:460] Webserver started at http://127.12.234.194:41409/ using document root <none> and password file <none>
I20260619 18:47:25.132233 16483 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:25.132306 16483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:25.132526 16483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:25.134765 16483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "5a38637492324f8cb0981999a17c6489"
format_stamp: "Formatted at 2026-06-19 18:47:25 on dist-test-slave-fljv"
server_key: "bcb53d4c956e6b8fc2441cf751d47f11"
server_key_iv: "87fb7bd6b1acd4e50748a214ffc8d4fd"
server_key_version: "encryptionkey@0"
I20260619 18:47:25.135349 16483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "5a38637492324f8cb0981999a17c6489"
format_stamp: "Formatted at 2026-06-19 18:47:25 on dist-test-slave-fljv"
server_key: "bcb53d4c956e6b8fc2441cf751d47f11"
server_key_iv: "87fb7bd6b1acd4e50748a214ffc8d4fd"
server_key_version: "encryptionkey@0"
I20260619 18:47:25.139322 16483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 18:47:25.141923 16498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:25.143239 16483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:25.143353 16483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "5a38637492324f8cb0981999a17c6489"
format_stamp: "Formatted at 2026-06-19 18:47:25 on dist-test-slave-fljv"
server_key: "bcb53d4c956e6b8fc2441cf751d47f11"
server_key_iv: "87fb7bd6b1acd4e50748a214ffc8d4fd"
server_key_version: "encryptionkey@0"
I20260619 18:47:25.143458 16483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:25.160090 16483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:25.164053 16483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:25.164271 16483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:25.165030 16483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:25.166147 16483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:25.166246 16483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:25.166320 16483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:25.166355 16483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:25.178438 16483 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:42457
I20260619 18:47:25.178468 16611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:42457 every 8 connection(s)
I20260619 18:47:25.179667 16483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:25.185832 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 16483
I20260619 18:47:25.185976 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:25.186352 13227 external_mini_cluster.cc:1428] Setting key 969f1766bf4441a5e86e36dd7bfe553b
Jun 19 18:47:25 dist-test-slave-fljv krb5kdc[16242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894845, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:47:25.197135 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:25.182397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:36939 (local address 127.12.234.254:35969)
0619 18:47:25.182572 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:25.182577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:25.183133 (+ 556us) server_negotiation.cc:407] Connection header received
0619 18:47:25.184722 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:25.184727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:25.184796 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:25.184897 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:25.186677 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.187462 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.189250 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.189606 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.189688 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:25.192952 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:25.192957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:25.192960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:25.192989 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:25.194633 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:25.195336 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:25.195340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:25.195342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:25.195438 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:25.195837 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:25.195841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:25.195843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:25.196077 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:25.196178 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:25.196847 (+ 669us) server_negotiation.cc:299] Negotiation successful
0619 18:47:25.196976 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 18:47:25.198378 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:25.182410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35969 (local address 127.12.234.194:36939)
0619 18:47:25.182971 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:25.183029 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:25.184426 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:25.185132 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:25.185143 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:25.185695 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:25.186463 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:25.186485 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.187640 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.187643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:25.189107 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:25.189118 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.189903 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.189907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:25.190118 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:25.190784 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:25.190809 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:25.192788 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:25.194743 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:25.194750 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:25.194769 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:25.195182 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:25.195536 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:25.195539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:25.195541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:25.195743 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:25.196199 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:25.196207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:25.196557 (+ 350us) client_negotiation.cc:769] Sending connection context
0619 18:47:25.196774 (+ 217us) client_negotiation.cc:240] Negotiation successful
0619 18:47:25.197166 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:47:25.199749 16612 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35969
I20260619 18:47:25.200038 16612 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:25.200664 16612 heartbeater.cc:511] Master 127.12.234.254:35969 requested a full tablet report, sending...
I20260619 18:47:25.201874 16287 ts_manager.cc:194] Registered new tserver with Master: 5a38637492324f8cb0981999a17c6489 (127.12.234.194:42457)
I20260619 18:47:25.202714 16287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:36939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:25.217061 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35969
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:33217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:25.329403 16617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:25.329707 16617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:25.329752 16617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:25.334057 16617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:25.334136 16617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:25.334232 16617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:25.339702 16617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:33217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:25.340759 16617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:25.341799 16617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:25.351416 16622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:25.351440 16625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:25.351608 16617 server_base.cc:1034] running on GCE node
W20260619 18:47:25.351421 16623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:25.352171 16617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:25.352797 16617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:25.353948 16617 hybrid_clock.cc:648] HybridClock initialized: now 1781894845353935 us; error 35 us; skew 500 ppm
Jun 19 18:47:25 dist-test-slave-fljv krb5kdc[16242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894845, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:25.356719 16617 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:25.357596 16617 webserver.cc:460] Webserver started at http://127.12.234.195:38115/ using document root <none> and password file <none>
I20260619 18:47:25.358122 16617 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:25.358210 16617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:25.358407 16617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:25.360455 16617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "4e3fb9186ec84f2e80fa9523025b4df8"
format_stamp: "Formatted at 2026-06-19 18:47:25 on dist-test-slave-fljv"
server_key: "72aca99b70297062cbc42987ce6d34cc"
server_key_iv: "399008545fd94d22653e4eb63647f6db"
server_key_version: "encryptionkey@0"
I20260619 18:47:25.361066 16617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "4e3fb9186ec84f2e80fa9523025b4df8"
format_stamp: "Formatted at 2026-06-19 18:47:25 on dist-test-slave-fljv"
server_key: "72aca99b70297062cbc42987ce6d34cc"
server_key_iv: "399008545fd94d22653e4eb63647f6db"
server_key_version: "encryptionkey@0"
I20260619 18:47:25.364909 16617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260619 18:47:25.367641 16632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:25.368945 16617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:25.369063 16617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "4e3fb9186ec84f2e80fa9523025b4df8"
format_stamp: "Formatted at 2026-06-19 18:47:25 on dist-test-slave-fljv"
server_key: "72aca99b70297062cbc42987ce6d34cc"
server_key_iv: "399008545fd94d22653e4eb63647f6db"
server_key_version: "encryptionkey@0"
I20260619 18:47:25.369186 16617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:25.383126 16617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:25.386286 16617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:25.386466 16617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:25.387087 16617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:25.387931 16617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:25.387979 16617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:25.388031 16617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:25.388058 16617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:25.396430 16617 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:43341
I20260619 18:47:25.396448 16745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:43341 every 8 connection(s)
I20260619 18:47:25.397317 16617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:25.403656 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 16617
I20260619 18:47:25.403764 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:25.404021 13227 external_mini_cluster.cc:1428] Setting key 588683b15a035a48e1ee03ade4471ee6
Jun 19 18:47:25 dist-test-slave-fljv krb5kdc[16242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894845, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:25.411058 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:25.399195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:60767 (local address 127.12.234.254:35969)
0619 18:47:25.399340 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:25.399344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:25.400264 (+ 920us) server_negotiation.cc:407] Connection header received
0619 18:47:25.401600 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:25.401603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:25.401663 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:25.401734 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:25.403003 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.403556 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.404842 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.405053 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.405138 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:25.407685 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:25.407688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:25.407690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:25.407715 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:25.409074 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:25.409567 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:25.409570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:25.409571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:25.409655 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:25.409978 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:25.409982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:25.409984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:25.410251 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:25.410329 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:25.410815 (+ 486us) server_negotiation.cc:299] Negotiation successful
0619 18:47:25.410918 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 18:47:25.411862 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:25.399591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35969 (local address 127.12.234.195:60767)
0619 18:47:25.400045 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:25.400138 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:25.401379 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:25.401932 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:25.401940 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:25.402361 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:25.402859 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:25.402873 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.403676 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.403679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:25.404730 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:25.404740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.405186 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.405190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:25.405348 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:25.405958 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:25.405977 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:25.407545 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:25.409173 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:25.409178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:25.409193 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:25.409459 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:25.409730 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:25.409731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:25.409733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:25.409874 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:25.410337 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:25.410342 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:25.410615 (+ 273us) client_negotiation.cc:769] Sending connection context
0619 18:47:25.410783 (+ 168us) client_negotiation.cc:240] Negotiation successful
0619 18:47:25.411056 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":110,"threads_started":1}
I20260619 18:47:25.413034 16746 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35969
I20260619 18:47:25.413281 16746 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:25.413810 16746 heartbeater.cc:511] Master 127.12.234.254:35969 requested a full tablet report, sending...
I20260619 18:47:25.414866 16287 ts_manager.cc:194] Registered new tserver with Master: 4e3fb9186ec84f2e80fa9523025b4df8 (127.12.234.195:43341)
I20260619 18:47:25.415428 16287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:60767
I20260619 18:47:25.417737 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 18:47:25 dist-test-slave-fljv krb5kdc[16242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894845, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 18:47:25 dist-test-slave-fljv krb5kdc[16242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894845, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:25.442966 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:25.434507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.12.234.254:35969)
0619 18:47:25.434639 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:25.434644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:25.434842 (+ 198us) server_negotiation.cc:407] Connection header received
0619 18:47:25.434975 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:25.434978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:25.435029 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:25.435104 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:25.435840 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.436387 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.437243 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:25.437477 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:25.437573 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:25.439876 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:25.439879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:25.439881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:25.439911 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:25.441304 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:25.441731 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:25.441734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:25.441736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:25.441820 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:25.442069 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:25.442073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:25.442074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:25.442354 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:25.442477 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:25.442696 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 18:47:25.442810 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260619 18:47:25.443336 16287 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53256
I20260619 18:47:25.444247 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 16349
I20260619 18:47:25.449155 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 16483
I20260619 18:47:25.453684 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 16617
I20260619 18:47:25.458467 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 16255
2026-06-19T18:47:25Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2037 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:25 dist-test-slave-fljv krb5kdc[16758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:25 dist-test-slave-fljv krb5kdc[16758](info): set up 2 sockets
Jun 19 18:47:25 dist-test-slave-fljv krb5kdc[16758](info): commencing operation
krb5kdc: starting...
W20260619 18:47:26.422019 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:26 dist-test-slave-fljv krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:26.541743 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:45723
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:45723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:26.652678 16771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:26.652963 16771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:26.653021 16771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:26.657136 16771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:26.657203 16771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:26.657230 16771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:26.657250 16771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:26.657272 16771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:26.662746 16771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:45723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:45723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:26.663805 16771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:26.664830 16771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:26.673717 16777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:26.673738 16776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:26.673717 16779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:26.674075 16771 server_base.cc:1034] running on GCE node
I20260619 18:47:26.674887 16771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:26.676139 16771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:26.677306 16771 hybrid_clock.cc:648] HybridClock initialized: now 1781894846677288 us; error 36 us; skew 500 ppm
Jun 19 18:47:26 dist-test-slave-fljv krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894846, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:26.684275 16771 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:26.685253 16771 webserver.cc:460] Webserver started at http://127.12.234.254:37051/ using document root <none> and password file <none>
I20260619 18:47:26.685829 16771 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:26.685899 16771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:26.686120 16771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:26.688267 16771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "09cb319f2b7b41f2b4f93cb743f20df4"
format_stamp: "Formatted at 2026-06-19 18:47:26 on dist-test-slave-fljv"
server_key: "055d5c5cb778853ee1d7327e34cd50a7"
server_key_iv: "3665880051558a17888132bc3f0546bd"
server_key_version: "encryptionkey@0"
I20260619 18:47:26.688828 16771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "09cb319f2b7b41f2b4f93cb743f20df4"
format_stamp: "Formatted at 2026-06-19 18:47:26 on dist-test-slave-fljv"
server_key: "055d5c5cb778853ee1d7327e34cd50a7"
server_key_iv: "3665880051558a17888132bc3f0546bd"
server_key_version: "encryptionkey@0"
I20260619 18:47:26.692639 16771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260619 18:47:26.695140 16786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:26.696487 16771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:26.696612 16771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "09cb319f2b7b41f2b4f93cb743f20df4"
format_stamp: "Formatted at 2026-06-19 18:47:26 on dist-test-slave-fljv"
server_key: "055d5c5cb778853ee1d7327e34cd50a7"
server_key_iv: "3665880051558a17888132bc3f0546bd"
server_key_version: "encryptionkey@0"
I20260619 18:47:26.696712 16771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:26.715890 16771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:26.719219 16771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:26.719427 16771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:26.726446 16771 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:45723
I20260619 18:47:26.726454 16838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:45723 every 8 connection(s)
I20260619 18:47:26.727470 16771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:26.728142 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 16771
I20260619 18:47:26.728235 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:26.728487 13227 external_mini_cluster.cc:1428] Setting key 2f7776769d52af14cbfd18541ee77a8d
I20260619 18:47:26.730650 16839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:26.736186 16839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4: Bootstrap starting.
Jun 19 18:47:26 dist-test-slave-fljv krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:26.738520 16839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:26.739248 16839 log.cc:824] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:26.741335 16839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4: No bootstrap required, opened a new log
I20260619 18:47:26.742085 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:26.729907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.12.234.254:45723)
0619 18:47:26.730369 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:26.730385 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:26.730421 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:47:26.731094 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:26.731112 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:26.731466 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:26.731786 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:26.732589 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:26.733599 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:26.734473 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:26.734951 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:26.735101 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:26.737577 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:26.737582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:26.737595 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:26.737625 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:26.739271 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:26.739764 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:26.739770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:26.739776 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:26.739915 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:26.740194 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:26.740196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:26.740197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:26.740586 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:26.740721 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:26.741002 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 18:47:26.741322 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":139,"threads_started":1}
I20260619 18:47:26.744736 16839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09cb319f2b7b41f2b4f93cb743f20df4" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45723 } }
I20260619 18:47:26.744963 16839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:26.745023 16839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09cb319f2b7b41f2b4f93cb743f20df4, State: Initialized, Role: FOLLOWER
I20260619 18:47:26.745417 16839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09cb319f2b7b41f2b4f93cb743f20df4" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45723 } }
I20260619 18:47:26.745540 16839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:26.745584 16839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:26.745674 16839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:26.746604 16839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09cb319f2b7b41f2b4f93cb743f20df4" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45723 } }
I20260619 18:47:26.746953 16839 leader_election.cc:302] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09cb319f2b7b41f2b4f93cb743f20df4; no voters:
I20260619 18:47:26.747256 16839 leader_election.cc:288] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:26.747370 16844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:26.747545 16844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [term 1 LEADER]: Becoming Leader. State: Replica: 09cb319f2b7b41f2b4f93cb743f20df4, State: Running, Role: LEADER
I20260619 18:47:26.747843 16844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09cb319f2b7b41f2b4f93cb743f20df4" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45723 } }
I20260619 18:47:26.748571 16839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:26.749529 16845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09cb319f2b7b41f2b4f93cb743f20df4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09cb319f2b7b41f2b4f93cb743f20df4" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45723 } } }
I20260619 18:47:26.749677 16845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:26.749922 16846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09cb319f2b7b41f2b4f93cb743f20df4. Latest consensus state: current_term: 1 leader_uuid: "09cb319f2b7b41f2b4f93cb743f20df4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09cb319f2b7b41f2b4f93cb743f20df4" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45723 } } }
I20260619 18:47:26.750005 16846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:26.750141 16853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:26.753300 16853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:26.758813 16853 catalog_manager.cc:1347] Generated new cluster ID: 0f8299f4e0e5400da1d59790a5fd34c8
I20260619 18:47:26.758955 16853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:26.776573 16853 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:26.777630 16853 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:26.782748 16853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09cb319f2b7b41f2b4f93cb743f20df4: Generated new TSK 0
I20260619 18:47:26.783468 16853 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:26.817332 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45723
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:26.952235 16865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:26.952497 16865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:26.952538 16865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:26.956700 16865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:26.956771 16865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:26.956830 16865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:26.962245 16865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:26.963413 16865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:26.964421 16865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:26.974009 16870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:26.974010 16873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:26.974009 16871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:26.974159 16865 server_base.cc:1034] running on GCE node
I20260619 18:47:26.974718 16865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:26.975399 16865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:26.976583 16865 hybrid_clock.cc:648] HybridClock initialized: now 1781894846976566 us; error 26 us; skew 500 ppm
Jun 19 18:47:26 dist-test-slave-fljv krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894846, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:26.979409 16865 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:26.980291 16865 webserver.cc:460] Webserver started at http://127.12.234.193:44193/ using document root <none> and password file <none>
I20260619 18:47:26.980847 16865 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:26.980912 16865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:26.981149 16865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:26.983287 16865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "15c1ab88231542f78e0f5d370ddb589a"
format_stamp: "Formatted at 2026-06-19 18:47:26 on dist-test-slave-fljv"
server_key: "c3d981530c52232b5c0b575ea1033bc4"
server_key_iv: "ae8f725be2843e939f14d4728e4151f1"
server_key_version: "encryptionkey@0"
I20260619 18:47:26.983831 16865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "15c1ab88231542f78e0f5d370ddb589a"
format_stamp: "Formatted at 2026-06-19 18:47:26 on dist-test-slave-fljv"
server_key: "c3d981530c52232b5c0b575ea1033bc4"
server_key_iv: "ae8f725be2843e939f14d4728e4151f1"
server_key_version: "encryptionkey@0"
I20260619 18:47:26.987860 16865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:26.990550 16880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:26.991875 16865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 18:47:26.992035 16865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "15c1ab88231542f78e0f5d370ddb589a"
format_stamp: "Formatted at 2026-06-19 18:47:26 on dist-test-slave-fljv"
server_key: "c3d981530c52232b5c0b575ea1033bc4"
server_key_iv: "ae8f725be2843e939f14d4728e4151f1"
server_key_version: "encryptionkey@0"
I20260619 18:47:26.992153 16865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:27.002734 16865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:27.005910 16865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:27.006101 16865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:27.006747 16865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:27.007597 16865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:27.007658 16865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:27.007715 16865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:27.007754 16865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:27.016287 16993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:45933 every 8 connection(s)
I20260619 18:47:27.017205 16865 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:45933
I20260619 18:47:27.018278 16865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:27.024523 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 16865
I20260619 18:47:27.024670 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:27.024933 13227 external_mini_cluster.cc:1428] Setting key e9f3ab792678090176217d748b2911ee
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894846, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:27.033772 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:27.020260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:58865 (local address 127.12.234.254:45723)
0619 18:47:27.020396 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:27.020399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:27.021334 (+ 935us) server_negotiation.cc:407] Connection header received
0619 18:47:27.022788 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:27.022791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:27.022852 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:27.022931 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:27.024218 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.024801 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.026136 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.026494 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.026605 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:27.029591 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:27.029594 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:27.029596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:27.029623 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:27.031277 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.031974 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.031978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.031979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.032122 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.032540 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.032543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.032544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.032745 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:27.032821 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:27.033513 (+ 692us) server_negotiation.cc:299] Negotiation successful
0619 18:47:27.033641 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:27.035035 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:27.020637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:45723 (local address 127.12.234.193:58865)
0619 18:47:27.021150 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:27.021200 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:27.022547 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:27.023135 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:27.023144 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:27.023537 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:27.024048 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:27.024063 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.024933 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.024936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:27.026005 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:27.026017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.026686 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.026690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:27.026886 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:27.027475 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:27.027492 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:27.029334 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:27.031421 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:27.031429 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:27.031453 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:27.031849 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:27.032222 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:27.032226 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:27.032228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:27.032431 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:27.032845 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:27.032852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:27.033231 (+ 379us) client_negotiation.cc:769] Sending connection context
0619 18:47:27.033464 (+ 233us) client_negotiation.cc:240] Negotiation successful
0619 18:47:27.033850 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":128,"threads_started":1}
I20260619 18:47:27.036288 16994 heartbeater.cc:347] Connected to a master server at 127.12.234.254:45723
I20260619 18:47:27.036540 16994 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:27.037081 16994 heartbeater.cc:511] Master 127.12.234.254:45723 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:47:27.038782 16803 ts_manager.cc:194] Registered new tserver with Master: 15c1ab88231542f78e0f5d370ddb589a (127.12.234.193:45933)
I20260619 18:47:27.040004 16803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:58865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:27.051347 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45723
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:27.161543 16999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:27.161818 16999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:27.161860 16999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:27.166019 16999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:27.166097 16999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:27.166157 16999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:27.171510 16999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:27.172595 16999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:27.173585 16999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:27.183655 17004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:27.183712 16999 server_base.cc:1034] running on GCE node
W20260619 18:47:27.183661 17007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:27.183650 17005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:27.184228 16999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:27.184841 16999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:27.186002 16999 hybrid_clock.cc:648] HybridClock initialized: now 1781894847185974 us; error 40 us; skew 500 ppm
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894847, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:27.188683 16999 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:27.189580 16999 webserver.cc:460] Webserver started at http://127.12.234.194:41285/ using document root <none> and password file <none>
I20260619 18:47:27.190102 16999 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:27.190160 16999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:27.190426 16999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:27.192606 16999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "269c421fac924a138c71c1e1a44f3aac"
format_stamp: "Formatted at 2026-06-19 18:47:27 on dist-test-slave-fljv"
server_key: "97822fc50bf643f14381f574ae5b621f"
server_key_iv: "827789eab02ddc29b20b64b9330089eb"
server_key_version: "encryptionkey@0"
I20260619 18:47:27.193207 16999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "269c421fac924a138c71c1e1a44f3aac"
format_stamp: "Formatted at 2026-06-19 18:47:27 on dist-test-slave-fljv"
server_key: "97822fc50bf643f14381f574ae5b621f"
server_key_iv: "827789eab02ddc29b20b64b9330089eb"
server_key_version: "encryptionkey@0"
I20260619 18:47:27.197057 16999 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260619 18:47:27.199610 17014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:27.201097 16999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:27.201211 16999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "269c421fac924a138c71c1e1a44f3aac"
format_stamp: "Formatted at 2026-06-19 18:47:27 on dist-test-slave-fljv"
server_key: "97822fc50bf643f14381f574ae5b621f"
server_key_iv: "827789eab02ddc29b20b64b9330089eb"
server_key_version: "encryptionkey@0"
I20260619 18:47:27.201313 16999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:27.220273 16999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:27.223438 16999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:27.223623 16999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:27.224217 16999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:27.225121 16999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:27.225175 16999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:27.225229 16999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:27.225255 16999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:27.233623 16999 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:42565
I20260619 18:47:27.233652 17127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:42565 every 8 connection(s)
I20260619 18:47:27.234514 16999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:27.237500 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 16999
I20260619 18:47:27.237594 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:27.237836 13227 external_mini_cluster.cc:1428] Setting key bda805ef21dc69db69abdf5e84714835
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894847, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:47:27.248409 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:27.236237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:33801 (local address 127.12.234.254:45723)
0619 18:47:27.236365 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:27.236369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:27.237213 (+ 844us) server_negotiation.cc:407] Connection header received
0619 18:47:27.238697 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:27.238700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:27.238749 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:27.238820 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:27.240028 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.240570 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.241715 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.241935 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.242029 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:27.244416 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:27.244419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:27.244421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:27.244446 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:27.245951 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.246740 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.246744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.246746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.246827 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.247192 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.247196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.247197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.247438 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:27.247526 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:27.248073 (+ 547us) server_negotiation.cc:299] Negotiation successful
0619 18:47:27.248232 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:47:27.249368 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:27.236627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:45723 (local address 127.12.234.194:33801)
0619 18:47:27.237065 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:27.237108 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:27.238496 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:27.238992 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:27.238999 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:27.239341 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:27.239847 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:27.239862 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.240692 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.240696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:27.241596 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:27.241605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.242074 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.242078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:27.242280 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:27.242842 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:27.242863 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:27.244255 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:27.246171 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:27.246300 (+ 129us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:27.246319 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:27.246624 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:27.246918 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:27.246922 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:27.246928 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:27.247087 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:27.247541 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:27.247548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:27.247836 (+ 288us) client_negotiation.cc:769] Sending connection context
0619 18:47:27.248001 (+ 165us) client_negotiation.cc:240] Negotiation successful
0619 18:47:27.248358 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":107,"threads_started":1}
I20260619 18:47:27.250715 17128 heartbeater.cc:347] Connected to a master server at 127.12.234.254:45723
I20260619 18:47:27.250989 17128 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:27.251631 17128 heartbeater.cc:511] Master 127.12.234.254:45723 requested a full tablet report, sending...
I20260619 18:47:27.252825 16803 ts_manager.cc:194] Registered new tserver with Master: 269c421fac924a138c71c1e1a44f3aac (127.12.234.194:42565)
I20260619 18:47:27.253683 16803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:33801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:27.266144 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45723
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:27.375389 17133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:27.375667 17133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:27.375726 17133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:27.379846 17133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:27.379916 17133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:27.380002 17133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:27.385303 17133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:27.386366 17133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:27.387379 17133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:47:27.397729 17133 server_base.cc:1034] running on GCE node
W20260619 18:47:27.397662 17139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:27.397660 17138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:27.397923 17141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:27.398417 17133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:27.399048 17133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:27.400199 17133 hybrid_clock.cc:648] HybridClock initialized: now 1781894847400176 us; error 35 us; skew 500 ppm
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894847, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:27.402984 17133 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:27.403844 17133 webserver.cc:460] Webserver started at http://127.12.234.195:35821/ using document root <none> and password file <none>
I20260619 18:47:27.404397 17133 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:27.404459 17133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:27.404675 17133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:27.406760 17133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "49c50c430b754e2dbd379d0c61ef19df"
format_stamp: "Formatted at 2026-06-19 18:47:27 on dist-test-slave-fljv"
server_key: "ae059738683ba663b08823bb7949e4b3"
server_key_iv: "656f928c8c88e041b9eafc372a887bb5"
server_key_version: "encryptionkey@0"
I20260619 18:47:27.407285 17133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "49c50c430b754e2dbd379d0c61ef19df"
format_stamp: "Formatted at 2026-06-19 18:47:27 on dist-test-slave-fljv"
server_key: "ae059738683ba663b08823bb7949e4b3"
server_key_iv: "656f928c8c88e041b9eafc372a887bb5"
server_key_version: "encryptionkey@0"
I20260619 18:47:27.410985 17133 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:47:27.413365 17148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:27.414682 17133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:27.414799 17133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "49c50c430b754e2dbd379d0c61ef19df"
format_stamp: "Formatted at 2026-06-19 18:47:27 on dist-test-slave-fljv"
server_key: "ae059738683ba663b08823bb7949e4b3"
server_key_iv: "656f928c8c88e041b9eafc372a887bb5"
server_key_version: "encryptionkey@0"
I20260619 18:47:27.414898 17133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:27.433404 17133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:27.436630 17133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:27.436813 17133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:27.437397 17133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:27.438215 17133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:27.438261 17133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:27.438310 17133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:27.438338 17133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:27.446568 17133 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:39583
I20260619 18:47:27.446615 17261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:39583 every 8 connection(s)
I20260619 18:47:27.447469 17133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:27.452574 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 17133
I20260619 18:47:27.452696 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:27.452948 13227 external_mini_cluster.cc:1428] Setting key 842fbd1242118c499aa209915363ce99
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894847, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:27.460873 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:27.449242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:49195 (local address 127.12.234.254:45723)
0619 18:47:27.449361 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:27.449365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:27.450305 (+ 940us) server_negotiation.cc:407] Connection header received
0619 18:47:27.451713 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:27.451717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:27.451773 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:27.451849 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:27.453133 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.453656 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.454812 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.455024 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.455108 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:27.457502 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:27.457505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:27.457507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:27.457533 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:27.458859 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.459356 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.459359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.459360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.459465 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.459797 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.459799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.459800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.460008 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:27.460091 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:27.460637 (+ 546us) server_negotiation.cc:299] Negotiation successful
0619 18:47:27.460752 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 18:47:27.461705 17264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:27.449659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:45723 (local address 127.12.234.195:49195)
0619 18:47:27.450116 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:27.450173 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:27.451514 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:27.452031 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:27.452038 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:27.452420 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:27.452994 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:27.453008 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.453773 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.453775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:27.454695 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:27.454704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.455151 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.455155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:27.455310 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:27.455861 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:27.455875 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:27.457329 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:27.458969 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:27.458973 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:27.458986 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:27.459237 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:27.459559 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:27.459561 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:27.459563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:27.459699 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:27.460122 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:27.460128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:27.460379 (+ 251us) client_negotiation.cc:769] Sending connection context
0619 18:47:27.460563 (+ 184us) client_negotiation.cc:240] Negotiation successful
0619 18:47:27.460827 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":109,"threads_started":1}
I20260619 18:47:27.462968 17262 heartbeater.cc:347] Connected to a master server at 127.12.234.254:45723
I20260619 18:47:27.463219 17262 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:27.463702 17262 heartbeater.cc:511] Master 127.12.234.254:45723 requested a full tablet report, sending...
I20260619 18:47:27.464852 16803 ts_manager.cc:194] Registered new tserver with Master: 49c50c430b754e2dbd379d0c61ef19df (127.12.234.195:39583)
I20260619 18:47:27.465548 16803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:49195
I20260619 18:47:27.466768 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:27.492481 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:27.482989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36786 (local address 127.12.234.193:45933)
0619 18:47:27.483342 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:27.483352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:27.483376 (+ 24us) server_negotiation.cc:407] Connection header received
0619 18:47:27.483424 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:27.483429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:27.483626 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:27.483741 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:27.484455 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.485245 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.486098 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.486556 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.486640 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:27.489067 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:27.489076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:27.489090 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:27.489116 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:27.490742 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.491152 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.491158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.491160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.491248 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.491489 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.491492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.491494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.491743 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:27.491943 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:27.492155 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 18:47:27.492301 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":158,"threads_started":1}
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:27.503476 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:27.495510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.12.234.254:45723)
0619 18:47:27.495623 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:27.495626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:27.495790 (+ 164us) server_negotiation.cc:407] Connection header received
0619 18:47:27.495922 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:27.495924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:27.495966 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:27.496052 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:27.496679 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.497217 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.498002 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:27.498289 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:27.498373 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:27.500623 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:27.500626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:27.500628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:27.500651 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:27.501927 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.502418 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.502421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.502422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.502507 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:27.502738 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:27.502739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:27.502740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:27.502929 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:27.503034 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:27.503254 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 18:47:27.503351 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 18:47:27.503834 16803 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42712
I20260619 18:47:27.504436 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 16865
I20260619 18:47:27.509466 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 16999
I20260619 18:47:27.514097 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 17133
I20260619 18:47:27.518639 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 16771
2026-06-19T18:47:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2059 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[17274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[17274](info): set up 2 sockets
Jun 19 18:47:27 dist-test-slave-fljv krb5kdc[17274](info): commencing operation
krb5kdc: starting...
W20260619 18:47:28.481966 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:28 dist-test-slave-fljv krb5kdc[17274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:28.601212 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40021
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:40021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:28.713881 17287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:28.714195 17287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:28.714239 17287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:28.718293 17287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:28.718354 17287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:28.718374 17287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:28.718389 17287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:28.718418 17287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:28.723968 17287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:40021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:28.725085 17287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:28.726130 17287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:28.734757 17292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:28.734755 17293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:28.734848 17287 server_base.cc:1034] running on GCE node
W20260619 18:47:28.734789 17295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:28.735668 17287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:28.736768 17287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:28.737941 17287 hybrid_clock.cc:648] HybridClock initialized: now 1781894848737916 us; error 33 us; skew 500 ppm
Jun 19 18:47:28 dist-test-slave-fljv krb5kdc[17274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894848, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:28.740805 17287 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:28.741561 17287 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:28.741638 17287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:28.741853 17287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:28.744046 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "9a2c5173947d42219c7950bb1b06d435"
format_stamp: "Formatted at 2026-06-19 18:47:28 on dist-test-slave-fljv"
server_key: "46047d8cee386a4faec7568e2d5a4ff8"
server_key_iv: "4e929f0fce1d781a54303362f2c6c312"
server_key_version: "encryptionkey@0"
I20260619 18:47:28.744623 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "9a2c5173947d42219c7950bb1b06d435"
format_stamp: "Formatted at 2026-06-19 18:47:28 on dist-test-slave-fljv"
server_key: "46047d8cee386a4faec7568e2d5a4ff8"
server_key_iv: "4e929f0fce1d781a54303362f2c6c312"
server_key_version: "encryptionkey@0"
I20260619 18:47:28.748553 17287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:28.751147 17301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:28.752542 17287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:28.752707 17287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "9a2c5173947d42219c7950bb1b06d435"
format_stamp: "Formatted at 2026-06-19 18:47:28 on dist-test-slave-fljv"
server_key: "46047d8cee386a4faec7568e2d5a4ff8"
server_key_iv: "4e929f0fce1d781a54303362f2c6c312"
server_key_version: "encryptionkey@0"
I20260619 18:47:28.752818 17287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:28.764782 17287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:28.768361 17287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:28.768566 17287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:28.776051 17287 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:40021
I20260619 18:47:28.776393 17353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:40021 every 8 connection(s)
I20260619 18:47:28.776790 17287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:28.777298 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 17287
I20260619 18:47:28.777424 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:28.777729 13227 external_mini_cluster.cc:1428] Setting key 6c2e57a6c412406584ed7ca4077065d2
I20260619 18:47:28.779958 17354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:28.786087 17354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435: Bootstrap starting.
Jun 19 18:47:28 dist-test-slave-fljv krb5kdc[17274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:28.788762 17354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:28.789526 17354 log.cc:824] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:28.791781 17354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435: No bootstrap required, opened a new log
I20260619 18:47:28.792205 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:28.779115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.12.234.254:40021)
0619 18:47:28.779547 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:28.779566 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:28.779781 (+ 215us) server_negotiation.cc:407] Connection header received
0619 18:47:28.780586 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:28.780605 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:28.780984 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:28.781327 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:28.782165 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:28.783225 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:28.784114 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:28.784606 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:28.784755 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:28.787228 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:28.787233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:28.787248 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:28.787279 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:28.789231 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:28.789719 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:28.789724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:28.789729 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:28.789849 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:28.790118 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:28.790121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:28.790122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:28.790588 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:28.790775 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:28.791076 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 18:47:28.791378 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":119,"threads_started":1}
I20260619 18:47:28.795956 17354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2c5173947d42219c7950bb1b06d435" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40021 } }
I20260619 18:47:28.796279 17354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:28.796375 17354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a2c5173947d42219c7950bb1b06d435, State: Initialized, Role: FOLLOWER
I20260619 18:47:28.796902 17354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2c5173947d42219c7950bb1b06d435" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40021 } }
I20260619 18:47:28.797046 17354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:28.797091 17354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:28.797171 17354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:28.798231 17354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2c5173947d42219c7950bb1b06d435" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40021 } }
I20260619 18:47:28.798575 17354 leader_election.cc:302] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a2c5173947d42219c7950bb1b06d435; no voters:
I20260619 18:47:28.798870 17354 leader_election.cc:288] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:28.799012 17359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:28.799219 17359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [term 1 LEADER]: Becoming Leader. State: Replica: 9a2c5173947d42219c7950bb1b06d435, State: Running, Role: LEADER
I20260619 18:47:28.799559 17359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2c5173947d42219c7950bb1b06d435" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40021 } }
I20260619 18:47:28.800125 17354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:28.801285 17361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a2c5173947d42219c7950bb1b06d435. Latest consensus state: current_term: 1 leader_uuid: "9a2c5173947d42219c7950bb1b06d435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2c5173947d42219c7950bb1b06d435" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40021 } } }
I20260619 18:47:28.801432 17361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:28.801839 17368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:28.801741 17360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a2c5173947d42219c7950bb1b06d435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2c5173947d42219c7950bb1b06d435" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40021 } } }
I20260619 18:47:28.802480 17360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:28.805028 17368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:28.810856 17368 catalog_manager.cc:1347] Generated new cluster ID: 30fa53fdbecc4a49a0a491dad06298a9
I20260619 18:47:28.810940 17368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:28.825037 17368 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:28.826113 17368 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:28.833380 17368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435: Generated new TSK 0
I20260619 18:47:28.834131 17368 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:28.873451 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40021
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:28.984807 17380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:28.985106 17380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:28.985154 17380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:28.989336 17380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:28.989408 17380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:28.989475 17380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:28.994967 17380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:28.996083 17380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:28.997120 17380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:29.006740 17386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:29.006744 17388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:29.006744 17385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:29.006821 17380 server_base.cc:1034] running on GCE node
I20260619 18:47:29.007508 17380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:29.008170 17380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:29.009343 17380 hybrid_clock.cc:648] HybridClock initialized: now 1781894849009321 us; error 33 us; skew 500 ppm
Jun 19 18:47:29 dist-test-slave-fljv krb5kdc[17274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894849, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:29.012173 17380 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:29.012915 17380 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:29.012979 17380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:29.013170 17380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:29.015265 17380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "91e90e3871a3470680d3bf737544ea92"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "505256b53004f6dd7e67299e82750e67"
server_key_iv: "226aa80441d20988d93aa27ec732419a"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.015796 17380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "91e90e3871a3470680d3bf737544ea92"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "505256b53004f6dd7e67299e82750e67"
server_key_iv: "226aa80441d20988d93aa27ec732419a"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.019369 17380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:47:29.021848 17394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.023310 17380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:29.023408 17380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "91e90e3871a3470680d3bf737544ea92"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "505256b53004f6dd7e67299e82750e67"
server_key_iv: "226aa80441d20988d93aa27ec732419a"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.023537 17380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:29.052677 17380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:29.055994 17380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:29.056255 17380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:29.056860 17380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:29.057780 17380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:29.057826 17380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.057878 17380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:29.057906 17380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.066258 17380 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:34689
I20260619 18:47:29.066295 17507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:34689 every 8 connection(s)
I20260619 18:47:29.066881 17380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:29.070286 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 17380
I20260619 18:47:29.070410 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:29.070695 13227 external_mini_cluster.cc:1428] Setting key 7a787c9f1a2edcf7544d03b4a85f244d
Jun 19 18:47:29 dist-test-slave-fljv krb5kdc[17274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894849, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:29.081457 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.068713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:48615 (local address 127.12.234.254:40021)
0619 18:47:29.068855 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.068859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.070069 (+ 1210us) server_negotiation.cc:407] Connection header received
0619 18:47:29.071296 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.071300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.071357 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.071436 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:29.072884 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.073429 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.074714 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.074941 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.075021 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.077859 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:29.077863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:29.077865 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:29.077892 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:29.079328 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:29.079965 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:29.079968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:29.079969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:29.080052 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:29.080392 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:29.080393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:29.080394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:29.080571 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:29.080646 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.081215 (+ 569us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.081328 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 18:47:29.082361 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.069092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40021 (local address 127.12.234.193:48615)
0619 18:47:29.069898 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.069944 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:29.071058 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:29.071635 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:29.071643 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:29.072027 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:29.072723 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.072742 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.073564 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.073568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.074597 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.074608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.075081 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.075085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.075283 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.075875 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:29.075897 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:29.077661 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:29.079467 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:29.079474 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:29.079493 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:29.079864 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:29.080157 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:29.080160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:29.080161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:29.080316 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:29.080678 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:29.080685 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:29.080963 (+ 278us) client_negotiation.cc:769] Sending connection context
0619 18:47:29.081150 (+ 187us) client_negotiation.cc:240] Negotiation successful
0619 18:47:29.081445 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:47:29.083742 17508 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40021
I20260619 18:47:29.083989 17508 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:29.084540 17508 heartbeater.cc:511] Master 127.12.234.254:40021 requested a full tablet report, sending...
I20260619 18:47:29.086323 17318 ts_manager.cc:194] Registered new tserver with Master: 91e90e3871a3470680d3bf737544ea92 (127.12.234.193:34689)
I20260619 18:47:29.087752 17318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:48615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:29.099454 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40021
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:29.211741 17513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:29.212013 17513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:29.212078 17513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:29.216259 17513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:29.216328 17513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:29.216413 17513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:29.221750 17513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:29.222841 17513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:29.223896 17513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:29.233287 17521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:29.233299 17518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:29.233346 17519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:29.233673 17513 server_base.cc:1034] running on GCE node
I20260619 18:47:29.234138 17513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:29.234810 17513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:29.235968 17513 hybrid_clock.cc:648] HybridClock initialized: now 1781894849235939 us; error 44 us; skew 500 ppm
Jun 19 18:47:29 dist-test-slave-fljv krb5kdc[17274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894849, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:29.238792 17513 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:29.239532 17513 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:29.239600 17513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:29.239789 17513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:29.241786 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "47302f1cc9e1480e8833c8425299166e"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "2549cf004be7d8db61a22c809536d8a4"
server_key_iv: "47e478edebcd6a860982c6936074ce98"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.242321 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "47302f1cc9e1480e8833c8425299166e"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "2549cf004be7d8db61a22c809536d8a4"
server_key_iv: "47e478edebcd6a860982c6936074ce98"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.245849 17513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:29.248279 17527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.249567 17513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:29.249679 17513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "47302f1cc9e1480e8833c8425299166e"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "2549cf004be7d8db61a22c809536d8a4"
server_key_iv: "47e478edebcd6a860982c6936074ce98"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.249782 17513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:29.267571 17513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:29.271068 17513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:29.271257 17513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:29.271878 17513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:29.272753 17513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:29.272800 17513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.272851 17513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:29.272881 17513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.281358 17513 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:41437
I20260619 18:47:29.281378 17640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:41437 every 8 connection(s)
I20260619 18:47:29.281937 17513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:29.286412 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 17513
I20260619 18:47:29.286530 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:29.286796 13227 external_mini_cluster.cc:1428] Setting key 0f63e52a61cdf2f14b8806aabf1cf28e
Jun 19 18:47:29 dist-test-slave-fljv krb5kdc[17274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894849, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:47:29.295644 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.283610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:52493 (local address 127.12.234.254:40021)
0619 18:47:29.283772 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.283776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.284982 (+ 1206us) server_negotiation.cc:407] Connection header received
0619 18:47:29.285854 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.285857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.285905 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.285984 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:29.287262 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.287795 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.288933 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.289239 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.289341 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.292250 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:29.292254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:29.292256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:29.292282 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:29.293644 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:29.294139 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:29.294142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:29.294143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:29.294250 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:29.294561 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:29.294564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:29.294566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:29.294747 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:29.294831 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.295338 (+ 507us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.295478 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 18:47:29.296568 17643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.284043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40021 (local address 127.12.234.194:52493)
0619 18:47:29.284826 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.284869 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:29.285644 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:29.286159 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:29.286166 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:29.286588 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:29.287123 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.287137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.287916 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.287920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.288811 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.288820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.289373 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.289376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.289585 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.290297 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:29.290317 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:29.292075 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:29.293757 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:29.293763 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:29.293781 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:29.294046 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:29.294331 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:29.294334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:29.294336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:29.294478 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:29.294843 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:29.294850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:29.295112 (+ 262us) client_negotiation.cc:769] Sending connection context
0619 18:47:29.295284 (+ 172us) client_negotiation.cc:240] Negotiation successful
0619 18:47:29.295633 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
I20260619 18:47:29.297679 17641 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:47:29.297923 17641 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:29.298409 17641 heartbeater.cc:511] Master 127.12.234.254:40021 requested a full tablet report, sending...
I20260619 18:47:29.299386 17318 ts_manager.cc:194] Registered new tserver with Master: 47302f1cc9e1480e8833c8425299166e (127.12.234.194:41437)
I20260619 18:47:29.300027 17318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:52493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:29.313373 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40021
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:29.425071 17646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:29.425356 17646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:29.425401 17646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:29.429836 17646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:29.429927 17646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:29.430015 17646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:29.435542 17646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:29.436640 17646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:29.437682 17646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:29.447556 17654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:29.447613 17651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:29.447695 17646 server_base.cc:1034] running on GCE node
W20260619 18:47:29.447582 17652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:29.448282 17646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:29.448927 17646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:29.450110 17646 hybrid_clock.cc:648] HybridClock initialized: now 1781894849450089 us; error 33 us; skew 500 ppm
Jun 19 18:47:29 dist-test-slave-fljv krb5kdc[17274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894849, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:29.453078 17646 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:29.453850 17646 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:29.453924 17646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:29.454133 17646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:29.456269 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "4e0c319a15084fa9be266e440d9ca8c9"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "3aaa208cd8327f68738c6b36ac489243"
server_key_iv: "16e5ff2161ba2c33a468b5db7bb3ffd6"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.456837 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "4e0c319a15084fa9be266e440d9ca8c9"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "3aaa208cd8327f68738c6b36ac489243"
server_key_iv: "16e5ff2161ba2c33a468b5db7bb3ffd6"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.460745 17646 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 18:47:29.463513 17660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.464919 17646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:29.465062 17646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "4e0c319a15084fa9be266e440d9ca8c9"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "3aaa208cd8327f68738c6b36ac489243"
server_key_iv: "16e5ff2161ba2c33a468b5db7bb3ffd6"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.465173 17646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:29.492532 17646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:29.495713 17646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:29.495899 17646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:29.496533 17646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:29.497401 17646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:29.497448 17646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.497500 17646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:29.497529 17646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.506283 17646 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:41359
I20260619 18:47:29.506312 17773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:41359 every 8 connection(s)
I20260619 18:47:29.506915 17646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:29.509644 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 17646
I20260619 18:47:29.509776 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:29.510068 13227 external_mini_cluster.cc:1428] Setting key 10800aa6f218554259a6411c8662b869
Jun 19 18:47:29 dist-test-slave-fljv krb5kdc[17274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894849, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:29.520320 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.508834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:46461 (local address 127.12.234.254:40021)
0619 18:47:29.508978 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.508982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.509982 (+ 1000us) server_negotiation.cc:407] Connection header received
0619 18:47:29.511374 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.511377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.511431 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.511508 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:29.512685 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.513210 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.514376 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.514588 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.514672 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.516973 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:29.516975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:29.516977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:29.517003 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:29.518380 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:29.518858 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:29.518862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:29.518864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:29.518950 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:29.519249 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:29.519252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:29.519254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:29.519432 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:29.519524 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.520021 (+ 497us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.520151 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 18:47:29.521288 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.509265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40021 (local address 127.12.234.195:46461)
0619 18:47:29.509822 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.509878 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:29.511173 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:29.511683 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:29.511691 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:29.512060 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:29.512556 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.512570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.513322 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.513325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.514266 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.514276 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.514717 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.514721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.514883 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.515421 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:29.515445 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:29.516807 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:29.518498 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:29.518504 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:29.518521 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:29.518768 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:29.519028 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:29.519031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:29.519033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:29.519167 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:29.519525 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:29.519532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:29.519795 (+ 263us) client_negotiation.cc:769] Sending connection context
0619 18:47:29.519965 (+ 170us) client_negotiation.cc:240] Negotiation successful
0619 18:47:29.520374 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":149,"threads_started":1}
I20260619 18:47:29.522605 17774 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40021
I20260619 18:47:29.522861 17774 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:29.523416 17774 heartbeater.cc:511] Master 127.12.234.254:40021 requested a full tablet report, sending...
I20260619 18:47:29.524593 17318 ts_manager.cc:194] Registered new tserver with Master: 4e0c319a15084fa9be266e440d9ca8c9 (127.12.234.195:41359)
I20260619 18:47:29.525178 17318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:46461
I20260619 18:47:29.536132 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:47:29.544936 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.538082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.12.234.254:40021)
0619 18:47:29.538230 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.538233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.538355 (+ 122us) server_negotiation.cc:407] Connection header received
0619 18:47:29.538448 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.538451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.538503 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.538562 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:29.539234 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.539775 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.540595 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.540800 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.540869 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.542128 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:29.542132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:29.542134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:29.542158 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:29.543408 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:29.543812 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:29.543815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:29.543817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:29.543902 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:29.544178 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:29.544180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:29.544181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:29.544391 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:29.544494 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.544708 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.544810 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:47:29.547875 17318 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:29.550863 17318 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:29.565405 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.559588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34689 (local address 127.0.0.1:51594)
0619 18:47:29.560218 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.560269 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:29.560418 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:29.561214 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:29.561220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:29.561240 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:29.561380 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.561387 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.562934 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.562937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.564046 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.564052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.565081 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.565085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.565164 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.565184 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 18:47:29.565233 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 18:47:29.565281 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"spinlock_wait_cycles":7296,"thread_start_us":86,"threads_started":1}
I20260619 18:47:29.565548 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.559708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.12.234.193:34689)
0619 18:47:29.560705 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.560712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.560736 (+ 24us) server_negotiation.cc:407] Connection header received
0619 18:47:29.560808 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.560813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.561050 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.561528 (+ 478us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:29.561674 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.562811 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.564180 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.564966 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.565104 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.565234 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.565285 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.565377 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"thread_start_us":90,"threads_started":1}
I20260619 18:47:29.565405 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.559765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:41359 (local address 127.0.0.1:38910)
0619 18:47:29.560283 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.560297 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:29.560417 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:29.561352 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:29.561355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:29.561372 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:29.561465 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.561472 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.562620 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.562623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.564023 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.564033 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.564833 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.564837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.564951 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.565017 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 18:47:29.565122 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 18:47:29.565240 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":3,"spinlock_wait_cycles":7168,"thread_start_us":115,"threads_started":1}
I20260619 18:47:29.566252 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.560059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38910 (local address 127.12.234.195:41359)
0619 18:47:29.560923 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.560935 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.560957 (+ 22us) server_negotiation.cc:407] Connection header received
0619 18:47:29.561036 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.561041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.561225 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.561371 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:29.561570 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.562503 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.564147 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.564726 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.564835 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.564982 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.566015 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.566102 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":89,"threads_started":1}
I20260619 18:47:29.566571 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.560900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53014 (local address 127.12.234.194:41437)
0619 18:47:29.561184 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.561195 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.561455 (+ 260us) server_negotiation.cc:407] Connection header received
0619 18:47:29.561528 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.561532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.561695 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.561833 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:29.562013 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.563075 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.564249 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.565091 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.565225 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.565324 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.566373 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.566452 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
I20260619 18:47:29.566783 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.560636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:41437 (local address 127.0.0.1:53014)
0619 18:47:29.561278 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.561290 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:29.561363 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:29.561824 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:29.561827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:29.561844 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:29.561924 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.561930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.563210 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.563214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.564138 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.564148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.565380 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.565384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.565467 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.565489 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:47:29.565563 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 18:47:29.565623 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":85,"threads_started":1}
I20260619 18:47:29.568578 17442 tablet_service.cc:1467] Processing CreateTablet for tablet 3a550e442e30457d9c1600dd51316492 (DEFAULT_TABLE table=test-table [id=9cdce0a6d9c24ccca1e56ca9dd7d45b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:29.568816 17708 tablet_service.cc:1467] Processing CreateTablet for tablet 3a550e442e30457d9c1600dd51316492 (DEFAULT_TABLE table=test-table [id=9cdce0a6d9c24ccca1e56ca9dd7d45b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:29.569698 17708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a550e442e30457d9c1600dd51316492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:29.569711 17442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a550e442e30457d9c1600dd51316492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:29.569358 17575 tablet_service.cc:1467] Processing CreateTablet for tablet 3a550e442e30457d9c1600dd51316492 (DEFAULT_TABLE table=test-table [id=9cdce0a6d9c24ccca1e56ca9dd7d45b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:29.570273 17575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a550e442e30457d9c1600dd51316492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:29.575831 17791 tablet_bootstrap.cc:492] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: Bootstrap starting.
I20260619 18:47:29.577584 17792 tablet_bootstrap.cc:492] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e: Bootstrap starting.
I20260619 18:47:29.577584 17793 tablet_bootstrap.cc:492] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92: Bootstrap starting.
I20260619 18:47:29.578299 17791 tablet_bootstrap.cc:654] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:29.579409 17791 log.cc:824] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:29.579763 17792 tablet_bootstrap.cc:654] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:29.580039 17793 tablet_bootstrap.cc:654] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:29.580924 17792 log.cc:824] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:29.580976 17793 log.cc:824] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:29.581306 17791 tablet_bootstrap.cc:492] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: No bootstrap required, opened a new log
I20260619 18:47:29.581516 17791 ts_tablet_manager.cc:1397] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260619 18:47:29.582870 17792 tablet_bootstrap.cc:492] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e: No bootstrap required, opened a new log
I20260619 18:47:29.583102 17792 ts_tablet_manager.cc:1397] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:47:29.583256 17793 tablet_bootstrap.cc:492] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92: No bootstrap required, opened a new log
I20260619 18:47:29.583423 17793 ts_tablet_manager.cc:1397] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 18:47:29.584573 17791 raft_consensus.cc:348] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } }
I20260619 18:47:29.584803 17791 raft_consensus.cc:374] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:29.584852 17791 raft_consensus.cc:729] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0c319a15084fa9be266e440d9ca8c9, State: Initialized, Role: FOLLOWER
I20260619 18:47:29.585352 17791 consensus_queue.cc:260] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } }
I20260619 18:47:29.586385 17791 ts_tablet_manager.cc:1428] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260619 18:47:29.586738 17774 heartbeater.cc:503] Master 127.12.234.254:40021 was elected leader, sending a full tablet report...
I20260619 18:47:29.587072 17792 raft_consensus.cc:348] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } }
I20260619 18:47:29.587343 17792 raft_consensus.cc:374] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:29.587387 17792 raft_consensus.cc:729] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47302f1cc9e1480e8833c8425299166e, State: Initialized, Role: FOLLOWER
I20260619 18:47:29.587247 17793 raft_consensus.cc:348] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } }
I20260619 18:47:29.587447 17793 raft_consensus.cc:374] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:29.587484 17793 raft_consensus.cc:729] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e90e3871a3470680d3bf737544ea92, State: Initialized, Role: FOLLOWER
I20260619 18:47:29.587882 17792 consensus_queue.cc:260] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } }
I20260619 18:47:29.587882 17793 consensus_queue.cc:260] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } }
I20260619 18:47:29.588485 17508 heartbeater.cc:503] Master 127.12.234.254:40021 was elected leader, sending a full tablet report...
I20260619 18:47:29.588866 17793 ts_tablet_manager.cc:1428] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 18:47:29.588863 17792 ts_tablet_manager.cc:1428] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:47:29.589185 17641 heartbeater.cc:503] Master 127.12.234.254:40021 was elected leader, sending a full tablet report...
I20260619 18:47:29.592967 17799 raft_consensus.cc:482] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:29.593120 17799 raft_consensus.cc:504] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } }
I20260619 18:47:29.594132 17799 leader_election.cc:288] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e90e3871a3470680d3bf737544ea92 (127.12.234.193:34689), 4e0c319a15084fa9be266e440d9ca8c9 (127.12.234.195:41359)
I20260619 18:47:29.598296 17643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.594398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34689 (local address 127.12.234.194:54251)
0619 18:47:29.594522 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.594537 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:29.594620 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:29.594847 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:29.594849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:29.594863 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:29.594979 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.594986 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.595948 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.595951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.597131 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.597141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.597903 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.597906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.597987 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.598008 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 18:47:29.598060 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 18:47:29.598130 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260619 18:47:29.598299 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.594468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:54251 (local address 127.12.234.193:34689)
0619 18:47:29.594619 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.594624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.594642 (+ 18us) server_negotiation.cc:407] Connection header received
0619 18:47:29.594701 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.594704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.594756 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.594828 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:29.595083 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.595841 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.597247 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.597790 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.597912 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.598013 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.598088 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.598156 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 18:47:29.598510 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.594399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:41359 (local address 127.12.234.194:46167)
0619 18:47:29.595078 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.595090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:29.595171 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:29.595365 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:29.595367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:29.595376 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:29.595462 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.595466 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.596210 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.596213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.597343 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:29.597349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.598115 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.598119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:29.598221 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.598241 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 18:47:29.598306 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 18:47:29.598364 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":344,"thread_start_us":144,"threads_started":1}
I20260619 18:47:29.598512 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.594533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:46167 (local address 127.12.234.195:41359)
0619 18:47:29.594706 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.594710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.595163 (+ 453us) server_negotiation.cc:407] Connection header received
0619 18:47:29.595236 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.595239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.595284 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.595363 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:29.595522 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.596124 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.597455 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.598014 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.598119 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.598160 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.598328 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.598397 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 18:47:29.598902 17462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a550e442e30457d9c1600dd51316492" candidate_uuid: "47302f1cc9e1480e8833c8425299166e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e90e3871a3470680d3bf737544ea92" is_pre_election: true
I20260619 18:47:29.599212 17462 raft_consensus.cc:2454] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47302f1cc9e1480e8833c8425299166e in term 0.
I20260619 18:47:29.599212 17728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a550e442e30457d9c1600dd51316492" candidate_uuid: "47302f1cc9e1480e8833c8425299166e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0c319a15084fa9be266e440d9ca8c9" is_pre_election: true
I20260619 18:47:29.599467 17728 raft_consensus.cc:2454] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47302f1cc9e1480e8833c8425299166e in term 0.
I20260619 18:47:29.599747 17528 leader_election.cc:302] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47302f1cc9e1480e8833c8425299166e, 91e90e3871a3470680d3bf737544ea92; no voters:
I20260619 18:47:29.600039 17799 raft_consensus.cc:2781] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:29.600112 17799 raft_consensus.cc:482] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:29.600142 17799 raft_consensus.cc:3037] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:29.601293 17799 raft_consensus.cc:504] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } }
I20260619 18:47:29.601622 17799 leader_election.cc:288] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [CANDIDATE]: Term 1 election: Requested vote from peers 91e90e3871a3470680d3bf737544ea92 (127.12.234.193:34689), 4e0c319a15084fa9be266e440d9ca8c9 (127.12.234.195:41359)
I20260619 18:47:29.601989 17728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a550e442e30457d9c1600dd51316492" candidate_uuid: "47302f1cc9e1480e8833c8425299166e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0c319a15084fa9be266e440d9ca8c9"
I20260619 18:47:29.602021 17462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a550e442e30457d9c1600dd51316492" candidate_uuid: "47302f1cc9e1480e8833c8425299166e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e90e3871a3470680d3bf737544ea92"
I20260619 18:47:29.602113 17728 raft_consensus.cc:3037] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:29.602123 17462 raft_consensus.cc:3037] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:29.603140 17462 raft_consensus.cc:2454] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47302f1cc9e1480e8833c8425299166e in term 1.
I20260619 18:47:29.603144 17728 raft_consensus.cc:2454] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47302f1cc9e1480e8833c8425299166e in term 1.
I20260619 18:47:29.603402 17528 leader_election.cc:302] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47302f1cc9e1480e8833c8425299166e, 91e90e3871a3470680d3bf737544ea92; no voters:
I20260619 18:47:29.603679 17799 raft_consensus.cc:2781] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:29.603885 17799 raft_consensus.cc:686] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 1 LEADER]: Becoming Leader. State: Replica: 47302f1cc9e1480e8833c8425299166e, State: Running, Role: LEADER
I20260619 18:47:29.604202 17799 consensus_queue.cc:237] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } }
I20260619 18:47:29.607371 17317 catalog_manager.cc:5495] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e reported cstate change: term changed from 0 to 1, leader changed from <none> to 47302f1cc9e1480e8833c8425299166e (127.12.234.194). New cstate: current_term: 1 leader_uuid: "47302f1cc9e1480e8833c8425299166e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47302f1cc9e1480e8833c8425299166e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41437 } health_report { overall_health: HEALTHY } } }
I20260619 18:47:29.619834 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.616316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53018 (local address 127.12.234.194:41437)
0619 18:47:29.616454 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.616458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.616471 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:47:29.616623 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.616625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.616679 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.616771 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:29.617018 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.617510 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.618405 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:29.618623 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:29.618708 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:29.619003 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:29.619440 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:29.619550 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.619618 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.619677 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 18:47:29.625844 17462 raft_consensus.cc:1261] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92 [term 1 FOLLOWER]: Refusing update from remote peer 47302f1cc9e1480e8833c8425299166e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:29.625846 17728 raft_consensus.cc:1261] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [term 1 FOLLOWER]: Refusing update from remote peer 47302f1cc9e1480e8833c8425299166e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:29.626547 17799 consensus_queue.cc:1035] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e90e3871a3470680d3bf737544ea92" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:29.626746 17801 consensus_queue.cc:1035] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0c319a15084fa9be266e440d9ca8c9" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:29.636289 17806 mvcc.cc:205] Tried to move back new op lower bound from 7298641304059588608 to 7298641303981424640. Current Snapshot: MvccSnapshot[applied={T|T < 7298641304059588608}]
I20260619 18:47:29.642169 17317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:29.642364 17317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:29.644686 17317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a2c5173947d42219c7950bb1b06d435: Sending DeleteTablet for 3 replicas of tablet 3a550e442e30457d9c1600dd51316492
I20260619 18:47:29.645404 17442 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a550e442e30457d9c1600dd51316492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:29 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:51594
I20260619 18:47:29.645433 17708 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a550e442e30457d9c1600dd51316492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:29 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:38910
I20260619 18:47:29.645488 17575 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a550e442e30457d9c1600dd51316492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:29 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:53014
I20260619 18:47:29.646154 17816 tablet_replica.cc:330] T 3a550e442e30457d9c1600dd51316492 P 91e90e3871a3470680d3bf737544ea92: stopping tablet replica
I20260619 18:47:29.646227 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 17380
I20260619 18:47:29.650384 17814 tablet_replica.cc:330] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e: stopping tablet replica
I20260619 18:47:29.650728 17815 tablet_replica.cc:330] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: stopping tablet replica
I20260619 18:47:29.650731 17814 raft_consensus.cc:2229] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:29.651083 17814 raft_consensus.cc:2258] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:29.651108 17815 raft_consensus.cc:2229] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:29.651566 17815 raft_consensus.cc:2258] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:29.653005 17815 ts_tablet_manager.cc:1905] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:29.653007 17814 ts_tablet_manager.cc:1905] T 3a550e442e30457d9c1600dd51316492 P 47302f1cc9e1480e8833c8425299166e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 18:47:29.653539 17302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:34689 (error 108)
I20260619 18:47:29.654335 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 17513
I20260619 18:47:29.656284 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.655942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34689 (local address 127.0.0.1:51596)
0619 18:47:29.656093 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.656171 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 18:47:29.656440 17815 ts_tablet_manager.cc:1918] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 18:47:29.656461 17302 catalog_manager.cc:4584] TS 91e90e3871a3470680d3bf737544ea92 (127.12.234.193:34689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3a550e442e30457d9c1600dd51316492: Network error: Client connection negotiation failed: client connection to 127.12.234.193:34689: connect: Connection refused (error 111)
I20260619 18:47:29.656543 17815 log.cc:1196] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/3a550e442e30457d9c1600dd51316492
I20260619 18:47:29.656810 17815 ts_tablet_manager.cc:1939] T 3a550e442e30457d9c1600dd51316492 P 4e0c319a15084fa9be266e440d9ca8c9: Deleting consensus metadata
I20260619 18:47:29.657925 17304 catalog_manager.cc:4841] TS 4e0c319a15084fa9be266e440d9ca8c9 (127.12.234.195:41359): tablet 3a550e442e30457d9c1600dd51316492 (table test-table [id=9cdce0a6d9c24ccca1e56ca9dd7d45b6]) successfully deleted
I20260619 18:47:29.659718 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 17646
I20260619 18:47:29.660682 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.659885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:41437 (local address 127.0.0.1:53034)
0619 18:47:29.660566 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:29.660613 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.194:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610}
I20260619 18:47:29.664393 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 17287
2026-06-19T18:47:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2146 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T18:47:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:29Z Disabled control of system clock
I20260619 18:47:29.692895 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40787
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:40787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 18:47:29.802470 17821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:29.802762 17821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:29.802798 17821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:29.806849 17821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:29.806915 17821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:29.806931 17821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:29.806946 17821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:29.806960 17821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:29.811866 17821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:40787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:29.812959 17821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:29.814350 17821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:29.822969 17829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:29.822969 17826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:29.822994 17827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:29.823065 17821 server_base.cc:1034] running on GCE node
I20260619 18:47:29.823840 17821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:29.825119 17821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:29.826262 17821 hybrid_clock.cc:648] HybridClock initialized: now 1781894849826235 us; error 43 us; skew 500 ppm
I20260619 18:47:29.828200 17821 webserver.cc:460] Webserver started at http://127.12.234.254:40967/ using document root <none> and password file <none>
I20260619 18:47:29.828780 17821 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:29.828845 17821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:29.829041 17821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:29.831192 17821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "245973c567d44c52a87daf0ef000dfe2"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "3ef71e333191f8c0feede4a804c0aed8"
server_key_iv: "b67fb1353ac3ccfd147ce0e67868df81"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.831758 17821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "245973c567d44c52a87daf0ef000dfe2"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "3ef71e333191f8c0feede4a804c0aed8"
server_key_iv: "b67fb1353ac3ccfd147ce0e67868df81"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.835625 17821 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:47:29.838280 17835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:29.839658 17821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:29.839802 17821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "245973c567d44c52a87daf0ef000dfe2"
format_stamp: "Formatted at 2026-06-19 18:47:29 on dist-test-slave-fljv"
server_key: "3ef71e333191f8c0feede4a804c0aed8"
server_key_iv: "b67fb1353ac3ccfd147ce0e67868df81"
server_key_version: "encryptionkey@0"
I20260619 18:47:29.839970 17821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:29.844800 17821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:29.845468 17821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:29.845641 17821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:29.852293 17821 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:40787
I20260619 18:47:29.852316 17887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:40787 every 8 connection(s)
I20260619 18:47:29.853307 17821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:29.856575 17888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:29.858449 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 17821
I20260619 18:47:29.858541 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:29.858814 13227 external_mini_cluster.cc:1428] Setting key 14dd34191bbbd2ead4c7ce822eea84f2
I20260619 18:47:29.862883 17888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2: Bootstrap starting.
I20260619 18:47:29.864698 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:29.860289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.12.234.254:40787)
0619 18:47:29.860959 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:29.860976 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:29.861012 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:47:29.861790 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:29.861809 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:29.862258 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:29.862577 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:29.863393 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:29.863397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:29.863410 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:29.863420 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:29.863475 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:29.863491 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:29.863595 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:29.863689 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 18:47:29.863940 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":123,"threads_started":1}
I20260619 18:47:29.865372 17888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:29.866250 17888 log.cc:824] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:29.868283 17888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2: No bootstrap required, opened a new log
I20260619 18:47:29.872059 17888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245973c567d44c52a87daf0ef000dfe2" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40787 } }
I20260619 18:47:29.872334 17888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:29.872375 17888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245973c567d44c52a87daf0ef000dfe2, State: Initialized, Role: FOLLOWER
I20260619 18:47:29.872766 17888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245973c567d44c52a87daf0ef000dfe2" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40787 } }
I20260619 18:47:29.872879 17888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:29.872925 17888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:29.872997 17888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:29.873931 17888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245973c567d44c52a87daf0ef000dfe2" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40787 } }
I20260619 18:47:29.874298 17888 leader_election.cc:302] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 245973c567d44c52a87daf0ef000dfe2; no voters:
I20260619 18:47:29.874598 17888 leader_election.cc:288] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:29.874701 17893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:29.874993 17893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [term 1 LEADER]: Becoming Leader. State: Replica: 245973c567d44c52a87daf0ef000dfe2, State: Running, Role: LEADER
I20260619 18:47:29.875298 17893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245973c567d44c52a87daf0ef000dfe2" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40787 } }
I20260619 18:47:29.875824 17888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:29.876856 17894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "245973c567d44c52a87daf0ef000dfe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245973c567d44c52a87daf0ef000dfe2" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40787 } } }
I20260619 18:47:29.876987 17894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:29.877345 17902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:29.877743 17895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 245973c567d44c52a87daf0ef000dfe2. Latest consensus state: current_term: 1 leader_uuid: "245973c567d44c52a87daf0ef000dfe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245973c567d44c52a87daf0ef000dfe2" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40787 } } }
I20260619 18:47:29.877842 17895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:29.880363 17902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:29.885917 17902 catalog_manager.cc:1347] Generated new cluster ID: 7a23eb2f23d24b65b3574303ecc5c92c
I20260619 18:47:29.886010 17902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:29.902637 17902 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:29.903026 17902 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:29.913218 17902 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2: Generated new TSK 0
I20260619 18:47:29.914155 17902 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:47:29.921172 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40787
--builtin_ntp_servers=127.12.234.212:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 18:47:30.034549 17912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:30.034837 17912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:30.034894 17912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:30.039011 17912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:30.039075 17912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:30.039155 17912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:30.044090 17912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:30.045194 17912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:30.046757 17912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:30.056530 17920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:30.056640 17912 server_base.cc:1034] running on GCE node
W20260619 18:47:30.056530 17918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:30.056646 17917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:30.057241 17912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:30.057873 17912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:30.059041 17912 hybrid_clock.cc:648] HybridClock initialized: now 1781894850058997 us; error 57 us; skew 500 ppm
I20260619 18:47:30.060935 17912 webserver.cc:460] Webserver started at http://127.12.234.193:41159/ using document root <none> and password file <none>
I20260619 18:47:30.061509 17912 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:30.061573 17912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:30.061761 17912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:30.063861 17912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "795a037554714c2f8a92777e6c5cf90d"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "279756bd5b34ceac7947fecacd6de551"
server_key_iv: "0b13f4764d7d3d91f6f810fad81d2be1"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.064390 17912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "795a037554714c2f8a92777e6c5cf90d"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "279756bd5b34ceac7947fecacd6de551"
server_key_iv: "0b13f4764d7d3d91f6f810fad81d2be1"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.068239 17912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:30.070971 17926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.072422 17912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:30.072557 17912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "795a037554714c2f8a92777e6c5cf90d"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "279756bd5b34ceac7947fecacd6de551"
server_key_iv: "0b13f4764d7d3d91f6f810fad81d2be1"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.072671 17912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:30.077550 17912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:30.078250 17912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:30.078418 17912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:30.078980 17912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:30.079917 17912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:30.079964 17912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.080016 17912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:30.080045 17912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.088761 17912 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:36141
I20260619 18:47:30.088788 18039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:36141 every 8 connection(s)
I20260619 18:47:30.089679 17912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:30.095837 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.091502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:55939 (local address 127.12.234.254:40787)
0619 18:47:30.091627 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.091630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.092736 (+ 1106us) server_negotiation.cc:407] Connection header received
0619 18:47:30.094084 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.094088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.094137 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.094234 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.095341 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.095346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.095348 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.095369 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.095389 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.095398 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.095496 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.095637 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.095696 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:47:30.096820 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.091873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40787 (local address 127.12.234.193:55939)
0619 18:47:30.092568 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.092610 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:30.093848 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:30.094408 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:30.094415 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:30.094515 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:30.095051 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:47:30.095052 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:30.095096 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:47:30.095176 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:30.095483 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:30.095492 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:30.095505 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 18:47:30.095680 (+ 175us) client_negotiation.cc:240] Negotiation successful
0619 18:47:30.095992 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":113,"threads_started":1}
I20260619 18:47:30.098059 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 17912
I20260619 18:47:30.098145 18040 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40787
I20260619 18:47:30.098212 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:30.098416 18040 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:30.098547 13227 external_mini_cluster.cc:1428] Setting key 0dbd7c97711ee486536dd4e0e747cf7b
I20260619 18:47:30.098930 18040 heartbeater.cc:511] Master 127.12.234.254:40787 requested a full tablet report, sending...
I20260619 18:47:30.100344 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40787
--builtin_ntp_servers=127.12.234.212:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 18:47:30.100447 17852 ts_manager.cc:194] Registered new tserver with Master: 795a037554714c2f8a92777e6c5cf90d (127.12.234.193:36141)
W20260619 18:47:30.212257 18043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:30.212567 18043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:30.212610 18043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:30.216666 18043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:30.216727 18043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:30.216784 18043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:30.221633 18043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:30.222750 18043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:30.224097 18043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:30.233596 18048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:30.233616 18049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:30.233619 18051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:30.233855 18043 server_base.cc:1034] running on GCE node
I20260619 18:47:30.234416 18043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:30.235087 18043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:30.236259 18043 hybrid_clock.cc:648] HybridClock initialized: now 1781894850236229 us; error 40 us; skew 500 ppm
I20260619 18:47:30.238149 18043 webserver.cc:460] Webserver started at http://127.12.234.194:37799/ using document root <none> and password file <none>
I20260619 18:47:30.238737 18043 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:30.238802 18043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:30.238996 18043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:30.241030 18043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "90d88ea443c6471f8bb56bce7dcb4cba"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "135278f281bf9d03698684f72e332a52"
server_key_iv: "3e5e8dcc024c3b78307d5c28a42749c0"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.241567 18043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "90d88ea443c6471f8bb56bce7dcb4cba"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "135278f281bf9d03698684f72e332a52"
server_key_iv: "3e5e8dcc024c3b78307d5c28a42749c0"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.245352 18043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:30.248122 18057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.249353 18043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 18:47:30.249459 18043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "90d88ea443c6471f8bb56bce7dcb4cba"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "135278f281bf9d03698684f72e332a52"
server_key_iv: "3e5e8dcc024c3b78307d5c28a42749c0"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.249580 18043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:30.254140 18043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:30.254799 18043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:30.254959 18043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:30.255478 18043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:30.256376 18043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:30.256419 18043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.256469 18043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:30.256503 18043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.264671 18043 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:34433
I20260619 18:47:30.264714 18170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:34433 every 8 connection(s)
I20260619 18:47:30.265579 18043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:30.266592 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 18043
I20260619 18:47:30.266710 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:30.266963 13227 external_mini_cluster.cc:1428] Setting key 397852d8ab95b72943acaedd04190078
I20260619 18:47:30.269098 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40787
--builtin_ntp_servers=127.12.234.212:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 18:47:30.271977 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.267557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:56163 (local address 127.12.234.254:40787)
0619 18:47:30.267692 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.267695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.268908 (+ 1213us) server_negotiation.cc:407] Connection header received
0619 18:47:30.270326 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.270329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.270373 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.270445 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.271535 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.271539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.271542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.271556 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.271574 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.271582 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.271667 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.271783 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.271828 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 18:47:30.272974 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.268034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40787 (local address 127.12.234.194:56163)
0619 18:47:30.268764 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.268806 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:30.270081 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:30.270618 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:30.270626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:30.270724 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:30.271278 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:47:30.271280 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:30.271322 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:47:30.271394 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:30.271647 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:30.271655 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:30.271661 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:47:30.271824 (+ 163us) client_negotiation.cc:240] Negotiation successful
0619 18:47:30.272146 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":113,"threads_started":1}
I20260619 18:47:30.274267 18171 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40787
I20260619 18:47:30.274518 18171 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:30.274925 18171 heartbeater.cc:511] Master 127.12.234.254:40787 requested a full tablet report, sending...
I20260619 18:47:30.276042 17852 ts_manager.cc:194] Registered new tserver with Master: 90d88ea443c6471f8bb56bce7dcb4cba (127.12.234.194:34433)
W20260619 18:47:30.382755 18174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:30.383041 18174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:30.383081 18174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:30.387140 18174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:30.387204 18174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:30.387262 18174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:30.392123 18174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:30.393208 18174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:30.394601 18174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:30.404053 18182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:30.404053 18179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:30.404055 18180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:30.404279 18174 server_base.cc:1034] running on GCE node
I20260619 18:47:30.404852 18174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:30.405512 18174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:30.406685 18174 hybrid_clock.cc:648] HybridClock initialized: now 1781894850406663 us; error 38 us; skew 500 ppm
I20260619 18:47:30.408563 18174 webserver.cc:460] Webserver started at http://127.12.234.195:46135/ using document root <none> and password file <none>
I20260619 18:47:30.409121 18174 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:30.409186 18174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:30.409374 18174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:30.411722 18174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "4a3da2c431f34b79baeccef89e099709"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "44c19d0f1c5e72566df8e14a5f167804"
server_key_iv: "a48b8a39720cc50bea832314a96f6ac1"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.412271 18174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "4a3da2c431f34b79baeccef89e099709"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "44c19d0f1c5e72566df8e14a5f167804"
server_key_iv: "a48b8a39720cc50bea832314a96f6ac1"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.416080 18174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:30.418708 18188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.420090 18174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:30.420212 18174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "4a3da2c431f34b79baeccef89e099709"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "44c19d0f1c5e72566df8e14a5f167804"
server_key_iv: "a48b8a39720cc50bea832314a96f6ac1"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.420326 18174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:30.425058 18174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:30.425689 18174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:30.425861 18174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:30.426484 18174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:30.427417 18174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:30.427464 18174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.427524 18174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:30.427553 18174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.436316 18174 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:37483
I20260619 18:47:30.436434 18301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:37483 every 8 connection(s)
I20260619 18:47:30.437269 18174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:30.443943 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.439226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:35173 (local address 127.12.234.254:40787)
0619 18:47:30.439350 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.439354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.440565 (+ 1211us) server_negotiation.cc:407] Connection header received
0619 18:47:30.442028 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.442032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.442085 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.442204 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.443424 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.443428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.443431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.443448 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.443473 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.443484 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.443589 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.443725 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.443776 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:47:30.444833 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.439627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40787 (local address 127.12.234.195:35173)
0619 18:47:30.440388 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.440431 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:30.441732 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:30.442391 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:30.442413 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:30.442514 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:30.443123 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:47:30.443125 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:30.443171 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:47:30.443246 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:30.443565 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:30.443575 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:30.443580 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 18:47:30.443752 (+ 172us) client_negotiation.cc:240] Negotiation successful
0619 18:47:30.444017 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":114,"threads_started":1}
I20260619 18:47:30.445381 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 18174
I20260619 18:47:30.445502 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:30.445811 13227 external_mini_cluster.cc:1428] Setting key 6eebb7253674587c47d2cb60753c522e
I20260619 18:47:30.446127 18302 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40787
I20260619 18:47:30.446396 18302 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:30.446883 18302 heartbeater.cc:511] Master 127.12.234.254:40787 requested a full tablet report, sending...
I20260619 18:47:30.447814 17852 ts_manager.cc:194] Registered new tserver with Master: 4a3da2c431f34b79baeccef89e099709 (127.12.234.195:37483)
I20260619 18:47:30.460058 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:47:30.463778 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.462076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.12.234.254:40787)
0619 18:47:30.462297 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.462302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.462443 (+ 141us) server_negotiation.cc:407] Connection header received
0619 18:47:30.462567 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.462569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.462609 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.462670 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.463426 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.463429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.463431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.463443 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.463461 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.463469 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.463551 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.463602 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.463636 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 18:47:30.466725 17852 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:30.469786 17852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:30.482093 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.479323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:34433 (local address 127.0.0.1:50128)
0619 18:47:30.479965 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.479976 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:30.480065 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:30.480533 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:30.480540 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:30.480620 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:30.481198 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:47:30.481200 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:30.481220 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:47:30.481284 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:30.481760 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:30.481769 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:30.481775 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:47:30.481874 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 18:47:30.481946 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":92,"threads_started":1}
I20260619 18:47:30.482121 18318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.479487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.12.234.194:34433)
0619 18:47:30.479888 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.479900 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.480062 (+ 162us) server_negotiation.cc:407] Connection header received
0619 18:47:30.480148 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.480155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.480369 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.480520 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.481413 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.481420 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.481434 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.481446 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.481492 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.481514 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.481585 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.481908 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.481990 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":105,"threads_started":1}
I20260619 18:47:30.482283 18316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.478572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:36141 (local address 127.0.0.1:48150)
0619 18:47:30.479688 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.479744 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:30.479858 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:30.481225 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:30.481228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:30.481241 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:30.481612 (+ 371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:47:30.481614 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:30.481624 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:47:30.481650 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:30.482008 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:30.482011 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:30.482014 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 18:47:30.482083 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 18:47:30.482118 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":80,"threads_started":1}
I20260619 18:47:30.482576 18314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.478605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37483 (local address 127.0.0.1:45204)
0619 18:47:30.479756 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.479768 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:30.480148 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:30.481248 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:30.481250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:30.481264 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:30.481765 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:47:30.481767 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:30.481777 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:47:30.481801 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:30.482339 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:30.482342 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:30.482344 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 18:47:30.482443 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 18:47:30.482473 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1100,"thread_start_us":90,"threads_started":1}
I20260619 18:47:30.482656 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.479394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48150 (local address 127.12.234.193:36141)
0619 18:47:30.480117 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.480129 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.480153 (+ 24us) server_negotiation.cc:407] Connection header received
0619 18:47:30.480207 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.480213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.480396 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.480518 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.481771 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.481777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.481789 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.481801 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.481842 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.481878 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.481968 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.482439 (+ 471us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.482523 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":100,"threads_started":1}
I20260619 18:47:30.482867 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.478833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45204 (local address 127.12.234.195:37483)
0619 18:47:30.479343 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.479355 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.479849 (+ 494us) server_negotiation.cc:407] Connection header received
0619 18:47:30.480251 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.480256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.480386 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.480488 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.481937 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.481942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.481952 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.481965 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.482016 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.482054 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.482124 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.482476 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.482550 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":224,"threads_started":1}
I20260619 18:47:30.485283 17974 tablet_service.cc:1467] Processing CreateTablet for tablet b62fe38c7456447491ca33f7edf59315 (DEFAULT_TABLE table=test-table [id=9203f0ec5314432a88a8ad48723e46a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:30.485462 18236 tablet_service.cc:1467] Processing CreateTablet for tablet b62fe38c7456447491ca33f7edf59315 (DEFAULT_TABLE table=test-table [id=9203f0ec5314432a88a8ad48723e46a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:30.485462 18105 tablet_service.cc:1467] Processing CreateTablet for tablet b62fe38c7456447491ca33f7edf59315 (DEFAULT_TABLE table=test-table [id=9203f0ec5314432a88a8ad48723e46a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:30.486311 17974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62fe38c7456447491ca33f7edf59315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:30.486506 18236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62fe38c7456447491ca33f7edf59315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:30.486508 18105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62fe38c7456447491ca33f7edf59315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:30.492393 18319 tablet_bootstrap.cc:492] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d: Bootstrap starting.
I20260619 18:47:30.493448 18320 tablet_bootstrap.cc:492] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709: Bootstrap starting.
I20260619 18:47:30.494956 18319 tablet_bootstrap.cc:654] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:30.495172 18321 tablet_bootstrap.cc:492] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba: Bootstrap starting.
I20260619 18:47:30.495906 18320 tablet_bootstrap.cc:654] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:30.496143 18319 log.cc:824] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:30.496753 18320 log.cc:824] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:30.497320 18321 tablet_bootstrap.cc:654] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:30.498111 18321 log.cc:824] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:30.498493 18319 tablet_bootstrap.cc:492] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d: No bootstrap required, opened a new log
I20260619 18:47:30.498733 18319 ts_tablet_manager.cc:1397] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 18:47:30.499894 18320 tablet_bootstrap.cc:492] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709: No bootstrap required, opened a new log
I20260619 18:47:30.500105 18320 ts_tablet_manager.cc:1397] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 18:47:30.503767 18319 raft_consensus.cc:348] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } }
I20260619 18:47:30.504097 18319 raft_consensus.cc:374] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:30.504151 18319 raft_consensus.cc:729] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795a037554714c2f8a92777e6c5cf90d, State: Initialized, Role: FOLLOWER
I20260619 18:47:30.504456 18321 tablet_bootstrap.cc:492] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba: No bootstrap required, opened a new log
I20260619 18:47:30.504688 18321 ts_tablet_manager.cc:1397] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260619 18:47:30.504696 18319 consensus_queue.cc:260] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } }
I20260619 18:47:30.504822 18320 raft_consensus.cc:348] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } }
I20260619 18:47:30.505095 18320 raft_consensus.cc:374] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:30.505154 18320 raft_consensus.cc:729] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a3da2c431f34b79baeccef89e099709, State: Initialized, Role: FOLLOWER
I20260619 18:47:30.505549 18040 heartbeater.cc:503] Master 127.12.234.254:40787 was elected leader, sending a full tablet report...
I20260619 18:47:30.505656 18320 consensus_queue.cc:260] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } }
I20260619 18:47:30.505818 18319 ts_tablet_manager.cc:1428] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260619 18:47:30.506376 18302 heartbeater.cc:503] Master 127.12.234.254:40787 was elected leader, sending a full tablet report...
I20260619 18:47:30.507637 18320 ts_tablet_manager.cc:1428] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260619 18:47:30.509600 18321 raft_consensus.cc:348] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } }
I20260619 18:47:30.509915 18321 raft_consensus.cc:374] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:30.509974 18321 raft_consensus.cc:729] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d88ea443c6471f8bb56bce7dcb4cba, State: Initialized, Role: FOLLOWER
I20260619 18:47:30.510461 18321 consensus_queue.cc:260] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } }
I20260619 18:47:30.511411 18321 ts_tablet_manager.cc:1428] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260619 18:47:30.511747 18171 heartbeater.cc:503] Master 127.12.234.254:40787 was elected leader, sending a full tablet report...
W20260619 18:47:30.516570 18172 tablet.cc:2369] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:47:30.523432 18326 raft_consensus.cc:482] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:30.523663 18326 raft_consensus.cc:504] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } }
I20260619 18:47:30.524989 18326 leader_election.cc:288] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 795a037554714c2f8a92777e6c5cf90d (127.12.234.193:36141), 90d88ea443c6471f8bb56bce7dcb4cba (127.12.234.194:34433)
I20260619 18:47:30.526813 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.525171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:36141 (local address 127.12.234.195:37751)
0619 18:47:30.525308 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.525324 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:30.525411 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:30.525760 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:30.525763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:30.525778 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:30.526250 (+ 472us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:47:30.526251 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:30.526262 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:47:30.526284 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:30.526544 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:30.526548 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:30.526552 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 18:47:30.526639 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 18:47:30.526686 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 18:47:30.526818 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.525303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:37751 (local address 127.12.234.193:36141)
0619 18:47:30.525443 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.525448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.525463 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:30.525508 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.525511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.525603 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.525684 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.526398 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.526402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.526405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.526418 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.526441 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.526451 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.526539 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.526634 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.526687 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 18:47:30.527324 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.525494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:34433 (local address 127.12.234.195:54351)
0619 18:47:30.525786 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.525797 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:30.525873 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:30.526144 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:30.526147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:30.526160 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:30.526752 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:47:30.526754 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:30.526763 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:47:30.526785 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:30.527087 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:30.527090 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:30.527093 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 18:47:30.527162 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 18:47:30.527192 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":115,"threads_started":1}
I20260619 18:47:30.527351 18318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.525597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:54351 (local address 127.12.234.194:34433)
0619 18:47:30.525709 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.525714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.525859 (+ 145us) server_negotiation.cc:407] Connection header received
0619 18:47:30.525985 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.525988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.526022 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.526089 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.526906 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.526910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.526913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.526926 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.526950 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.526959 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.527046 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.527195 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.527238 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:47:30.527459 17994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b62fe38c7456447491ca33f7edf59315" candidate_uuid: "4a3da2c431f34b79baeccef89e099709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795a037554714c2f8a92777e6c5cf90d" is_pre_election: true
I20260619 18:47:30.527750 17994 raft_consensus.cc:2454] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a3da2c431f34b79baeccef89e099709 in term 0.
I20260619 18:47:30.528173 18190 leader_election.cc:302] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a3da2c431f34b79baeccef89e099709, 795a037554714c2f8a92777e6c5cf90d; no voters:
I20260619 18:47:30.528439 18326 raft_consensus.cc:2781] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:30.528517 18326 raft_consensus.cc:482] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:30.528560 18326 raft_consensus.cc:3037] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:30.528524 18125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b62fe38c7456447491ca33f7edf59315" candidate_uuid: "4a3da2c431f34b79baeccef89e099709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" is_pre_election: true
I20260619 18:47:30.528787 18125 raft_consensus.cc:2454] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a3da2c431f34b79baeccef89e099709 in term 0.
I20260619 18:47:30.529729 18326 raft_consensus.cc:504] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } }
I20260619 18:47:30.530067 18326 leader_election.cc:288] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [CANDIDATE]: Term 1 election: Requested vote from peers 795a037554714c2f8a92777e6c5cf90d (127.12.234.193:36141), 90d88ea443c6471f8bb56bce7dcb4cba (127.12.234.194:34433)
I20260619 18:47:30.530421 17994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b62fe38c7456447491ca33f7edf59315" candidate_uuid: "4a3da2c431f34b79baeccef89e099709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795a037554714c2f8a92777e6c5cf90d"
I20260619 18:47:30.530462 18125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b62fe38c7456447491ca33f7edf59315" candidate_uuid: "4a3da2c431f34b79baeccef89e099709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90d88ea443c6471f8bb56bce7dcb4cba"
I20260619 18:47:30.530565 18125 raft_consensus.cc:3037] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:30.530565 17994 raft_consensus.cc:3037] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:30.531855 17994 raft_consensus.cc:2454] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a3da2c431f34b79baeccef89e099709 in term 1.
I20260619 18:47:30.531855 18125 raft_consensus.cc:2454] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a3da2c431f34b79baeccef89e099709 in term 1.
I20260619 18:47:30.532158 18190 leader_election.cc:302] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a3da2c431f34b79baeccef89e099709, 90d88ea443c6471f8bb56bce7dcb4cba; no voters:
I20260619 18:47:30.532318 18326 raft_consensus.cc:2781] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:30.532523 18326 raft_consensus.cc:686] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 1 LEADER]: Becoming Leader. State: Replica: 4a3da2c431f34b79baeccef89e099709, State: Running, Role: LEADER
I20260619 18:47:30.532853 18326 consensus_queue.cc:237] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } }
I20260619 18:47:30.536115 17850 catalog_manager.cc:5495] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a3da2c431f34b79baeccef89e099709 (127.12.234.195). New cstate: current_term: 1 leader_uuid: "4a3da2c431f34b79baeccef89e099709" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a3da2c431f34b79baeccef89e099709" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 } health_report { overall_health: UNKNOWN } } }
I20260619 18:47:30.554888 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.553333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45206 (local address 127.12.234.195:37483)
0619 18:47:30.553455 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.553459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.553474 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:30.553581 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.553584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.553644 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.553734 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.554568 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.554571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.554574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.554585 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.554605 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.554613 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.554692 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.554733 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.554768 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:47:30.561342 17994 raft_consensus.cc:1261] T b62fe38c7456447491ca33f7edf59315 P 795a037554714c2f8a92777e6c5cf90d [term 1 FOLLOWER]: Refusing update from remote peer 4a3da2c431f34b79baeccef89e099709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:30.561342 18125 raft_consensus.cc:1261] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [term 1 FOLLOWER]: Refusing update from remote peer 4a3da2c431f34b79baeccef89e099709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:30.562086 18326 consensus_queue.cc:1035] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "795a037554714c2f8a92777e6c5cf90d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 36141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:30.562312 18329 consensus_queue.cc:1035] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90d88ea443c6471f8bb56bce7dcb4cba" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 34433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:30.569063 18332 mvcc.cc:205] Tried to move back new op lower bound from 7298641307890778112 to 7298641307785523200. Current Snapshot: MvccSnapshot[applied={T|T < 7298641307890778112}]
I20260619 18:47:30.579144 17850 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:30.579314 17850 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:30.581962 17850 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 245973c567d44c52a87daf0ef000dfe2: Sending DeleteTablet for 3 replicas of tablet b62fe38c7456447491ca33f7edf59315
I20260619 18:47:30.582988 17974 tablet_service.cc:1514] Processing DeleteTablet for tablet b62fe38c7456447491ca33f7edf59315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:30 UTC) from {username='slave'} at 127.0.0.1:48150
I20260619 18:47:30.583035 18236 tablet_service.cc:1514] Processing DeleteTablet for tablet b62fe38c7456447491ca33f7edf59315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:30 UTC) from {username='slave'} at 127.0.0.1:45204
I20260619 18:47:30.583443 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 17912
I20260619 18:47:30.583525 18105 tablet_service.cc:1514] Processing DeleteTablet for tablet b62fe38c7456447491ca33f7edf59315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:30 UTC) from {username='slave'} at 127.0.0.1:50128
W20260619 18:47:30.588898 17837 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:36141 (error 108)
I20260619 18:47:30.589123 18343 tablet_replica.cc:330] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba: stopping tablet replica
I20260619 18:47:30.589250 18342 tablet_replica.cc:330] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709: stopping tablet replica
I20260619 18:47:30.589522 18343 raft_consensus.cc:2229] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:30.589553 18342 raft_consensus.cc:2229] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:30.589916 18343 raft_consensus.cc:2258] T b62fe38c7456447491ca33f7edf59315 P 90d88ea443c6471f8bb56bce7dcb4cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:30.590106 18342 raft_consensus.cc:2258] T b62fe38c7456447491ca33f7edf59315 P 4a3da2c431f34b79baeccef89e099709 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:30.590271 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.589895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:36141 (local address 127.0.0.1:48162)
0619 18:47:30.590079 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.590147 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:36141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260619 18:47:30.590320 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 18043
W20260619 18:47:30.590950 17837 catalog_manager.cc:4584] TS 795a037554714c2f8a92777e6c5cf90d (127.12.234.193:36141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b62fe38c7456447491ca33f7edf59315: Network error: Client connection negotiation failed: client connection to 127.12.234.193:36141: connect: Connection refused (error 111)
I20260619 18:47:30.595332 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 18174
I20260619 18:47:30.597198 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.596903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:34433 (local address 127.0.0.1:50130)
0619 18:47:30.597046 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.597107 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.194:34433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 18:47:30.600612 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 17821
2026-06-19T18:47:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (935 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 18:47:30.621531 13227 mini_oidc.cc:152] Starting JWKS server
I20260619 18:47:30.623315 13227 webserver.cc:460] Webserver started at https://127.0.0.1:36325/ using document root <none> and password file <none>
I20260619 18:47:30.623488 13227 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 18:47:30.623744 13227 webserver.cc:460] Webserver started at http://127.0.0.1:44261/ using document root <none> and password file <none>
2026-06-19T18:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:30Z Disabled control of system clock
I20260619 18:47:30.632505 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:39995
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:39995
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36325/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/testchainca.cert with env {}
W20260619 18:47:30.743582 18350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:30.743878 18350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:30.743930 18350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:30.748078 18350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:30.748149 18350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:30.748164 18350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:30.748180 18350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:30.748194 18350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 18:47:30.748209 18350 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 18:47:30.748231 18350 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36325/jwks.json
I20260619 18:47:30.753813 18350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:39995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:39995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36325/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:30.754918 18350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:30.755892 18350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:30.764266 18358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:30.764261 18356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:30.764261 18355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:30.764575 18350 server_base.cc:1034] running on GCE node
I20260619 18:47:30.765268 18350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:30.766463 18350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:30.767655 18350 hybrid_clock.cc:648] HybridClock initialized: now 1781894850767630 us; error 45 us; skew 500 ppm
I20260619 18:47:30.769522 18350 webserver.cc:460] Webserver started at http://127.12.234.254:44863/ using document root <none> and password file <none>
I20260619 18:47:30.770083 18350 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:30.770141 18350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:30.770359 18350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:30.772406 18350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "36f2fda18c4f47b5bd780d34ebb7362c"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "986141167f13a12d2953a23150547c43"
server_key_iv: "63eb5e2b2a69131c8ab9e6a31b3904f1"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.772953 18350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "36f2fda18c4f47b5bd780d34ebb7362c"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "986141167f13a12d2953a23150547c43"
server_key_iv: "63eb5e2b2a69131c8ab9e6a31b3904f1"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.776641 18350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 18:47:30.779256 18364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:30.780565 18350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:30.780683 18350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "36f2fda18c4f47b5bd780d34ebb7362c"
format_stamp: "Formatted at 2026-06-19 18:47:30 on dist-test-slave-fljv"
server_key: "986141167f13a12d2953a23150547c43"
server_key_iv: "63eb5e2b2a69131c8ab9e6a31b3904f1"
server_key_version: "encryptionkey@0"
I20260619 18:47:30.780787 18350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:30.802889 18350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:30.803618 18350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:30.803893 18350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:30.810902 18350 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:39995
I20260619 18:47:30.810922 18416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:39995 every 8 connection(s)
I20260619 18:47:30.811900 18350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:30.815292 18417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:30.818557 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 18350
I20260619 18:47:30.818677 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:30.818914 13227 external_mini_cluster.cc:1428] Setting key b24b6b3c55398b070379881b7a7e5669
I20260619 18:47:30.821462 18417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c: Bootstrap starting.
I20260619 18:47:30.824409 18417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:30.825300 18417 log.cc:824] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:30.827612 18417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c: No bootstrap required, opened a new log
I20260619 18:47:30.829241 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.820248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56672 (local address 127.12.234.254:39995)
0619 18:47:30.820980 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.821001 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.821048 (+ 47us) server_negotiation.cc:407] Connection header received
0619 18:47:30.821927 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.821965 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.822541 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.822963 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:30.823430 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:30.824869 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:30.825786 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:30.826436 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:30.826579 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:30.827433 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:30.827438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:30.827468 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:30.827890 (+ 422us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:30.827947 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:30.827975 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:30.828059 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:30.828241 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 18:47:30.828538 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":104,"threads_started":1}
I20260619 18:47:30.831517 18417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f2fda18c4f47b5bd780d34ebb7362c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39995 } }
I20260619 18:47:30.831799 18417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:30.831859 18417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f2fda18c4f47b5bd780d34ebb7362c, State: Initialized, Role: FOLLOWER
I20260619 18:47:30.832243 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:47:30.832317 18417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f2fda18c4f47b5bd780d34ebb7362c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39995 } }
I20260619 18:47:30.832465 18417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:30.832525 18417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:30.832617 18417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:30.833664 18417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f2fda18c4f47b5bd780d34ebb7362c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39995 } }
I20260619 18:47:30.834044 18417 leader_election.cc:302] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36f2fda18c4f47b5bd780d34ebb7362c; no voters:
I20260619 18:47:30.834398 18417 leader_election.cc:288] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:30.834497 18422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:30.834708 18422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [term 1 LEADER]: Becoming Leader. State: Replica: 36f2fda18c4f47b5bd780d34ebb7362c, State: Running, Role: LEADER
I20260619 18:47:30.835003 18422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f2fda18c4f47b5bd780d34ebb7362c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39995 } }
I20260619 18:47:30.835542 18417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:30.836525 18424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36f2fda18c4f47b5bd780d34ebb7362c. Latest consensus state: current_term: 1 leader_uuid: "36f2fda18c4f47b5bd780d34ebb7362c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f2fda18c4f47b5bd780d34ebb7362c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39995 } } }
I20260619 18:47:30.836668 18424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:30.836948 18423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36f2fda18c4f47b5bd780d34ebb7362c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f2fda18c4f47b5bd780d34ebb7362c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39995 } } }
I20260619 18:47:30.837038 18431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:30.837051 18423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:30.840206 18431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:30.845700 18431 catalog_manager.cc:1347] Generated new cluster ID: 9645e4b0fee846b1a6d0f0edc9a6f54f
I20260619 18:47:30.845765 18431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:30.867040 18431 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:30.868307 18431 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:30.879634 18431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36f2fda18c4f47b5bd780d34ebb7362c: Generated new TSK 0
I20260619 18:47:30.880553 18431 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 18:47:30.912616 18447 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 18:47:30.912719 18447 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 18:47:30.913348 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:30.895881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.12.234.254:39995)
0619 18:47:30.896018 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:30.896023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:30.896169 (+ 146us) server_negotiation.cc:407] Connection header received
0619 18:47:30.896369 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:30.896373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:30.896437 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:30.896559 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0619 18:47:30.896773 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:30.897603 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:30.898517 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:30.898870 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:30.898976 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:30.899375 (+ 399us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 18:47:30.912911 (+ 13536us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 18:47:30.913208 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 18:47:30.913443 18420 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 18:47:30.914541 18446 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:30.895770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:39995 (local address 127.0.0.1:56682)
0619 18:47:30.896085 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:30.896098 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:30.896215 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:30.896558 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:30.896560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:30.896566 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0619 18:47:30.896678 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:30.896683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:30.897735 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:30.897738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:30.898405 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:30.898415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:30.898983 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:30.898986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:30.899115 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:30.899171 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 18:47:30.913166 (+ 13995us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 18:47:30.913367 (+ 201us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.234.254:39995: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.12.234.254:39995: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 18:47:30.915222 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 18350
2026-06-19T18:47:30Z chronyd exiting
I20260619 18:47:30.951642 13227 test_util.cc:175] -----------------------------------------------
I20260619 18:47:30.951730 13227 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781894826428600-13227-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (332 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 18:47:30.952777 13227 mini_oidc.cc:152] Starting JWKS server
I20260619 18:47:30.953621 13227 webserver.cc:460] Webserver started at https://127.0.0.1:39969/ using document root <none> and password file <none>
I20260619 18:47:30.953702 13227 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 18:47:30.953951 13227 webserver.cc:460] Webserver started at http://127.0.0.1:40249/ using document root <none> and password file <none>
2026-06-19T18:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:30Z Disabled control of system clock
I20260619 18:47:30.962459 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:36097
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:36097
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39969/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/testchainca.cert with env {}
W20260619 18:47:31.071762 18453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:31.072098 18453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:31.072158 18453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:31.076275 18453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:31.076346 18453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:31.076367 18453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:31.076390 18453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:31.076409 18453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 18:47:31.076435 18453 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 18:47:31.076462 18453 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39969/jwks.json
I20260619 18:47:31.082013 18453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:36097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:36097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39969/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:31.083235 18453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:31.084287 18453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:31.092919 18459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:31.092923 18458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:31.093171 18453 server_base.cc:1034] running on GCE node
W20260619 18:47:31.092919 18461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:31.093952 18453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:31.095137 18453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:31.096314 18453 hybrid_clock.cc:648] HybridClock initialized: now 1781894851096286 us; error 41 us; skew 500 ppm
I20260619 18:47:31.098336 18453 webserver.cc:460] Webserver started at http://127.12.234.254:35999/ using document root <none> and password file <none>
I20260619 18:47:31.098887 18453 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:31.098949 18453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:31.099156 18453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:31.101276 18453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "3c307d9a17664b438966a9045537b6fd"
format_stamp: "Formatted at 2026-06-19 18:47:31 on dist-test-slave-fljv"
server_key: "77751e4990872a058c015ea90749a868"
server_key_iv: "c086e2d50b83a4cf3203aa157a4d1cc1"
server_key_version: "encryptionkey@0"
I20260619 18:47:31.101847 18453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "3c307d9a17664b438966a9045537b6fd"
format_stamp: "Formatted at 2026-06-19 18:47:31 on dist-test-slave-fljv"
server_key: "77751e4990872a058c015ea90749a868"
server_key_iv: "c086e2d50b83a4cf3203aa157a4d1cc1"
server_key_version: "encryptionkey@0"
I20260619 18:47:31.105648 18453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:31.108799 18467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:31.110549 18453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:31.110726 18453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "3c307d9a17664b438966a9045537b6fd"
format_stamp: "Formatted at 2026-06-19 18:47:31 on dist-test-slave-fljv"
server_key: "77751e4990872a058c015ea90749a868"
server_key_iv: "c086e2d50b83a4cf3203aa157a4d1cc1"
server_key_version: "encryptionkey@0"
I20260619 18:47:31.110844 18453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:31.128383 18453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:31.129065 18453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:31.129218 18453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:31.135869 18453 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:36097
I20260619 18:47:31.135887 18519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:36097 every 8 connection(s)
I20260619 18:47:31.136915 18453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:31.138314 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 18453
I20260619 18:47:31.138414 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:31.138670 13227 external_mini_cluster.cc:1428] Setting key 5d5f3463baad002fa62b74832d638242
I20260619 18:47:31.140678 18520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:31.146920 18520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd: Bootstrap starting.
I20260619 18:47:31.148092 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:31.140014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35364 (local address 127.12.234.254:36097)
0619 18:47:31.140706 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:31.140722 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:31.140757 (+ 35us) server_negotiation.cc:407] Connection header received
0619 18:47:31.141427 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:31.141445 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:31.141868 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:31.142161 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:31.142566 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.143577 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.144752 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.145247 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.145367 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:31.146301 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:31.146307 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:31.146320 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:31.146729 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:31.146788 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:31.146808 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:31.146885 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:31.147049 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 18:47:31.147347 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":119,"threads_started":1}
I20260619 18:47:31.149214 18520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:31.149962 18520 log.cc:824] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:31.150838 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:47:31.151875 18520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd: No bootstrap required, opened a new log
I20260619 18:47:31.155365 18520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c307d9a17664b438966a9045537b6fd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36097 } }
I20260619 18:47:31.155611 18520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:31.155669 18520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c307d9a17664b438966a9045537b6fd, State: Initialized, Role: FOLLOWER
I20260619 18:47:31.156090 18520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c307d9a17664b438966a9045537b6fd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36097 } }
I20260619 18:47:31.156211 18520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:31.156272 18520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:31.156369 18520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:31.157270 18520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c307d9a17664b438966a9045537b6fd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36097 } }
I20260619 18:47:31.157596 18520 leader_election.cc:302] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c307d9a17664b438966a9045537b6fd; no voters:
I20260619 18:47:31.157889 18520 leader_election.cc:288] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:31.158064 18525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:31.158303 18525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [term 1 LEADER]: Becoming Leader. State: Replica: 3c307d9a17664b438966a9045537b6fd, State: Running, Role: LEADER
I20260619 18:47:31.158598 18525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c307d9a17664b438966a9045537b6fd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36097 } }
I20260619 18:47:31.159127 18520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:31.160158 18526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c307d9a17664b438966a9045537b6fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c307d9a17664b438966a9045537b6fd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36097 } } }
I20260619 18:47:31.160288 18526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:31.160588 18527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c307d9a17664b438966a9045537b6fd. Latest consensus state: current_term: 1 leader_uuid: "3c307d9a17664b438966a9045537b6fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c307d9a17664b438966a9045537b6fd" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36097 } } }
I20260619 18:47:31.160674 18527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd [sys.catalog]: This master's current role is: LEADER
W20260619 18:47:31.164002 18540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:47:31.164075 18540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:47:31.164153 18534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:31.164824 18534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:31.170233 18534 catalog_manager.cc:1347] Generated new cluster ID: 6a169c59c10249608ab6f4ae4e210b47
I20260619 18:47:31.170315 18534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:31.183419 18534 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:31.184746 18534 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:31.192019 18534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c307d9a17664b438966a9045537b6fd: Generated new TSK 0
I20260619 18:47:31.192922 18534 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 18:47:31.218621 18550 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 18:47:31.218703 18550 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 18:47:31.219452 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:31.203096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.12.234.254:36097)
0619 18:47:31.203300 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:31.203305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:31.203513 (+ 208us) server_negotiation.cc:407] Connection header received
0619 18:47:31.203707 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:31.203710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:31.203780 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:31.203908 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0619 18:47:31.204179 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.204998 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.205842 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.206205 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.206301 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:31.206634 (+ 333us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 18:47:31.219029 (+ 12395us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 18:47:31.219311 (+ 282us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35366: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":109}
W20260619 18:47:31.219517 18523 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35366: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 18:47:31.219553 18549 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:31.202917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:36097 (local address 127.0.0.1:35366)
0619 18:47:31.203417 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:31.203431 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:31.203550 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:31.203913 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:31.203915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:31.203921 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0619 18:47:31.204075 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:31.204081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.205122 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.205126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:31.205743 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:31.205751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.206343 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.206346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:31.206415 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:31.206442 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 18:47:31.219235 (+ 12793us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 18:47:31.219396 (+ 161us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.234.254:36097: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":212,"threads_started":1}
I20260619 18:47:31.220482 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 18453
2026-06-19T18:47:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (305 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 18:47:31.258638 13227 mini_oidc.cc:152] Starting JWKS server
I20260619 18:47:31.259509 13227 webserver.cc:460] Webserver started at https://127.0.0.1:46729/ using document root <none> and password file <none>
I20260619 18:47:31.259595 13227 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 18:47:31.259800 13227 webserver.cc:460] Webserver started at http://127.0.0.1:35505/ using document root <none> and password file <none>
2026-06-19T18:47:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:31Z Disabled control of system clock
I20260619 18:47:31.268767 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41301
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:35483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:41301
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46729/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 18:47:31.380048 18556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:31.380329 18556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:31.380368 18556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:31.384479 18556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:31.384545 18556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:31.384559 18556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:31.384575 18556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:31.384589 18556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 18:47:31.384604 18556 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 18:47:31.384616 18556 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46729/jwks.json
I20260619 18:47:31.389463 18556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:41301
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46729/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:31.390571 18556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:31.392019 18556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:47:31.400558 18556 server_base.cc:1034] running on GCE node
W20260619 18:47:31.400508 18561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:31.400508 18564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:31.400544 18562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:31.401363 18556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:31.402624 18556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:31.403800 18556 hybrid_clock.cc:648] HybridClock initialized: now 1781894851403780 us; error 32 us; skew 500 ppm
I20260619 18:47:31.405740 18556 webserver.cc:460] Webserver started at http://127.12.234.254:42927/ using document root <none> and password file <none>
I20260619 18:47:31.406354 18556 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:31.406425 18556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:31.406627 18556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:31.408736 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "515acb9c985440a187dca7f2a1f45d8c"
format_stamp: "Formatted at 2026-06-19 18:47:31 on dist-test-slave-fljv"
server_key: "e838eadb7f267b891d851140e8467ff7"
server_key_iv: "c0ceaa32673302fe9eb07fee65001450"
server_key_version: "encryptionkey@0"
I20260619 18:47:31.409286 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "515acb9c985440a187dca7f2a1f45d8c"
format_stamp: "Formatted at 2026-06-19 18:47:31 on dist-test-slave-fljv"
server_key: "e838eadb7f267b891d851140e8467ff7"
server_key_iv: "c0ceaa32673302fe9eb07fee65001450"
server_key_version: "encryptionkey@0"
I20260619 18:47:31.413180 18556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260619 18:47:31.415865 18570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:31.417340 18556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:31.417475 18556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "515acb9c985440a187dca7f2a1f45d8c"
format_stamp: "Formatted at 2026-06-19 18:47:31 on dist-test-slave-fljv"
server_key: "e838eadb7f267b891d851140e8467ff7"
server_key_iv: "c0ceaa32673302fe9eb07fee65001450"
server_key_version: "encryptionkey@0"
I20260619 18:47:31.417595 18556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:31.442132 18556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:31.442950 18556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:31.443156 18556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:31.449957 18556 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:41301
I20260619 18:47:31.450021 18622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:41301 every 8 connection(s)
I20260619 18:47:31.450982 18556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:31.454151 18623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:31.454427 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 18556
I20260619 18:47:31.454558 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:31.454797 13227 external_mini_cluster.cc:1428] Setting key c212c0f1550c51a337af3b6ac26c55dd
I20260619 18:47:31.460417 18623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c: Bootstrap starting.
I20260619 18:47:31.463079 18623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:31.463893 18623 log.cc:824] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:31.464430 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:31.456194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54512 (local address 127.12.234.254:41301)
0619 18:47:31.456805 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:31.456821 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:31.456858 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:47:31.457528 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:31.457546 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:31.457973 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:31.458313 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:31.458660 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.459629 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.460884 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.461389 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.461526 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:31.462452 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:31.462457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:31.462471 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:31.462891 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:31.462953 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:31.462973 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:31.463108 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:31.463289 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 18:47:31.463591 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":108,"threads_started":1}
I20260619 18:47:31.466126 18623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c: No bootstrap required, opened a new log
I20260619 18:47:31.467537 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:47:31.470045 18623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515acb9c985440a187dca7f2a1f45d8c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41301 } }
I20260619 18:47:31.470329 18623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:31.470388 18623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 515acb9c985440a187dca7f2a1f45d8c, State: Initialized, Role: FOLLOWER
I20260619 18:47:31.470839 18623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515acb9c985440a187dca7f2a1f45d8c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41301 } }
I20260619 18:47:31.470975 18623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:31.471041 18623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:31.471138 18623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:31.472074 18623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515acb9c985440a187dca7f2a1f45d8c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41301 } }
I20260619 18:47:31.472401 18623 leader_election.cc:302] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 515acb9c985440a187dca7f2a1f45d8c; no voters:
I20260619 18:47:31.472784 18623 leader_election.cc:288] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:31.472880 18628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:31.473124 18628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [term 1 LEADER]: Becoming Leader. State: Replica: 515acb9c985440a187dca7f2a1f45d8c, State: Running, Role: LEADER
I20260619 18:47:31.473454 18628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515acb9c985440a187dca7f2a1f45d8c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41301 } }
I20260619 18:47:31.474056 18623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:31.475303 18629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "515acb9c985440a187dca7f2a1f45d8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515acb9c985440a187dca7f2a1f45d8c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41301 } } }
I20260619 18:47:31.475385 18630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 515acb9c985440a187dca7f2a1f45d8c. Latest consensus state: current_term: 1 leader_uuid: "515acb9c985440a187dca7f2a1f45d8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515acb9c985440a187dca7f2a1f45d8c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41301 } } }
I20260619 18:47:31.475437 18629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:31.475451 18630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:31.475816 18638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:31.478730 18638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:31.484527 18638 catalog_manager.cc:1347] Generated new cluster ID: dfe1f05ece884418bee078720e71cfda
I20260619 18:47:31.484606 18638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:31.494616 18638 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:31.495680 18638 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:31.500133 18638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 515acb9c985440a187dca7f2a1f45d8c: Generated new TSK 0
I20260619 18:47:31.501025 18638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:47:31.524417 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:31.519112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54528 (local address 127.12.234.254:41301)
0619 18:47:31.519259 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:31.519263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:31.519444 (+ 181us) server_negotiation.cc:407] Connection header received
0619 18:47:31.519627 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:31.519630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:31.519687 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:31.519785 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0619 18:47:31.520043 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.520604 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.521915 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.522222 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.522311 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:31.522681 (+ 370us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 18:47:31.524047 (+ 1366us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 18:47:31.524278 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54528: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 18:47:31.524492 18626 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54528: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 18:47:31.526420 18652 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:31.519047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41301 (local address 127.0.0.1:54528)
0619 18:47:31.519348 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:31.519364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:31.519502 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:31.519802 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:31.519805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:31.519814 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0619 18:47:31.519937 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:31.519943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.520712 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.520716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:31.521766 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:31.521779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.522337 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.522341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:31.522433 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:31.522473 (+ 40us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 18:47:31.524215 (+ 1742us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 18:47:31.526235 (+ 2020us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.234.254:41301: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":1778,"thread_start_us":102,"threads_started":1}
I20260619 18:47:31.526876 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 18556
E20260619 18:47:31.535461 18653 webserver.cc:533] Webserver: sslize failed:
E20260619 18:47:31.535542 18653 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T18:47:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (306 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T18:47:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:31Z Disabled control of system clock
I20260619 18:47:31.573360 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:34573
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:34573
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/testchainca.cert with env {}
W20260619 18:47:31.684725 18657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:31.685016 18657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:31.685057 18657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:31.689152 18657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:31.689225 18657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:31.689239 18657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:31.689256 18657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:31.689270 18657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 18:47:31.689284 18657 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 18:47:31.689296 18657 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 18:47:31.694906 18657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:34573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:34573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:31.696048 18657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:31.697094 18657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:31.705823 18665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:31.705802 18662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:31.705802 18663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:31.706023 18657 server_base.cc:1034] running on GCE node
I20260619 18:47:31.706779 18657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:31.707926 18657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:31.709103 18657 hybrid_clock.cc:648] HybridClock initialized: now 1781894851709081 us; error 37 us; skew 500 ppm
I20260619 18:47:31.716651 18657 webserver.cc:460] Webserver started at http://127.12.234.254:36427/ using document root <none> and password file <none>
I20260619 18:47:31.717293 18657 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:31.717370 18657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:31.717595 18657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:31.719825 18657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "065aaf3729b2467495802ac53e7afa61"
format_stamp: "Formatted at 2026-06-19 18:47:31 on dist-test-slave-fljv"
server_key: "4287e23375673124ce1b35fa72c8863d"
server_key_iv: "9a84bb301ea7a417b533cd2ce66e7e3a"
server_key_version: "encryptionkey@0"
I20260619 18:47:31.720419 18657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "065aaf3729b2467495802ac53e7afa61"
format_stamp: "Formatted at 2026-06-19 18:47:31 on dist-test-slave-fljv"
server_key: "4287e23375673124ce1b35fa72c8863d"
server_key_iv: "9a84bb301ea7a417b533cd2ce66e7e3a"
server_key_version: "encryptionkey@0"
I20260619 18:47:31.724417 18657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:31.727200 18671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:31.728736 18657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:31.728868 18657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "065aaf3729b2467495802ac53e7afa61"
format_stamp: "Formatted at 2026-06-19 18:47:31 on dist-test-slave-fljv"
server_key: "4287e23375673124ce1b35fa72c8863d"
server_key_iv: "9a84bb301ea7a417b533cd2ce66e7e3a"
server_key_version: "encryptionkey@0"
I20260619 18:47:31.728986 18657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:31.746424 18657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:31.747157 18657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:31.747335 18657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:31.753978 18657 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:34573
I20260619 18:47:31.753990 18723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:34573 every 8 connection(s)
I20260619 18:47:31.754988 18657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:31.757869 18724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:31.759243 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 18657
I20260619 18:47:31.759347 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:31.759598 13227 external_mini_cluster.cc:1428] Setting key 68adc8195f4d1b0ee4311fd058e2ac17
I20260619 18:47:31.763970 18724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61: Bootstrap starting.
I20260619 18:47:31.766618 18724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:31.767493 18724 log.cc:824] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:31.769040 18727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:31.760926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.12.234.254:34573)
0619 18:47:31.761561 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:31.761577 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:31.761615 (+ 38us) server_negotiation.cc:407] Connection header received
0619 18:47:31.762324 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:31.762349 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:31.762770 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:31.763054 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:31.763387 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.764351 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.765545 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.766051 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.766171 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:31.767021 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:31.767027 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:31.767043 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:47:31.767542 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:31.767625 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:47:31.767652 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:31.767744 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:31.767851 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 18:47:31.768138 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":123,"threads_started":1}
I20260619 18:47:31.769661 18724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61: No bootstrap required, opened a new log
I20260619 18:47:31.771718 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:47:31.773375 18724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065aaf3729b2467495802ac53e7afa61" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34573 } }
I20260619 18:47:31.773610 18724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:31.773664 18724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 065aaf3729b2467495802ac53e7afa61, State: Initialized, Role: FOLLOWER
I20260619 18:47:31.774098 18724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065aaf3729b2467495802ac53e7afa61" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34573 } }
I20260619 18:47:31.774257 18724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:31.774320 18724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:31.774417 18724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:31.775302 18724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065aaf3729b2467495802ac53e7afa61" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34573 } }
I20260619 18:47:31.775635 18724 leader_election.cc:302] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 065aaf3729b2467495802ac53e7afa61; no voters:
I20260619 18:47:31.775945 18724 leader_election.cc:288] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:31.776069 18729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:31.776265 18729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [term 1 LEADER]: Becoming Leader. State: Replica: 065aaf3729b2467495802ac53e7afa61, State: Running, Role: LEADER
I20260619 18:47:31.776579 18729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065aaf3729b2467495802ac53e7afa61" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34573 } }
I20260619 18:47:31.777137 18724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:31.778214 18731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 065aaf3729b2467495802ac53e7afa61. Latest consensus state: current_term: 1 leader_uuid: "065aaf3729b2467495802ac53e7afa61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065aaf3729b2467495802ac53e7afa61" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34573 } } }
I20260619 18:47:31.778211 18730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "065aaf3729b2467495802ac53e7afa61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "065aaf3729b2467495802ac53e7afa61" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34573 } } }
I20260619 18:47:31.778365 18730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:31.778365 18731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:31.778780 18738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:31.781880 18738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:31.787490 18738 catalog_manager.cc:1347] Generated new cluster ID: c896e7ded99d43018bb32c6b452c6f4c
I20260619 18:47:31.787572 18738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:31.807165 18738 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:31.808483 18738 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:31.817414 18738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 065aaf3729b2467495802ac53e7afa61: Generated new TSK 0
I20260619 18:47:31.818148 18738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:47:31.944625 18727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:31.831054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43656 (local address 127.12.234.254:34573)
0619 18:47:31.831196 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:31.831200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:31.831396 (+ 196us) server_negotiation.cc:407] Connection header received
0619 18:47:31.831530 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:31.831532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:31.831583 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:31.831689 (+ 106us) server_negotiation.cc:226] Negotiated authn=JWT
0619 18:47:31.831928 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.832518 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.833808 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.834050 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.834142 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:31.834514 (+ 372us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 18:47:31.944231 (+109717us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 18:47:31.944502 (+ 271us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43656: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 18:47:31.944706 18727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43656: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 18:47:31.944801 18753 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:31.830971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:34573 (local address 127.0.0.1:43656)
0619 18:47:31.831274 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:31.831290 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:31.831411 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:31.831697 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:31.831700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:31.831708 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0619 18:47:31.831822 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:31.831829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.832620 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.832623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:31.833667 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:31.833679 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:31.834199 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:31.834203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:31.834297 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:31.834328 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 18:47:31.944466 (+110138us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 18:47:31.944649 (+ 183us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.234.254:34573: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":89,"threads_started":1}
I20260619 18:47:31.945415 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 18657
2026-06-19T18:47:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (398 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (122 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (122 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:32 dist-test-slave-fljv krb5kdc[18765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:32 dist-test-slave-fljv krb5kdc[18765](info): set up 2 sockets
Jun 19 18:47:32 dist-test-slave-fljv krb5kdc[18765](info): commencing operation
krb5kdc: starting...
W20260619 18:47:33.150210 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:33 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:33.271422 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:42293
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:42293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:33.383554 18778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:33.383862 18778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:33.383903 18778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:33.388145 18778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:33.388227 18778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:33.388247 18778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:33.388262 18778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:33.388276 18778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:33.394032 18778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:42293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:42293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:33.395202 18778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:33.396260 18778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:33.404970 18783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:33.405081 18778 server_base.cc:1034] running on GCE node
W20260619 18:47:33.404979 18784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:33.405027 18786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:33.405767 18778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:33.407027 18778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:33.408212 18778 hybrid_clock.cc:648] HybridClock initialized: now 1781894853408194 us; error 36 us; skew 500 ppm
Jun 19 18:47:33 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894853, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:33.411271 18778 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:33.412250 18778 webserver.cc:460] Webserver started at http://127.12.234.254:40089/ using document root <none> and password file <none>
I20260619 18:47:33.412810 18778 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:33.412873 18778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:33.413079 18778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:33.415230 18778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "572ee59e5ae442b69026a7ede5834911"
format_stamp: "Formatted at 2026-06-19 18:47:33 on dist-test-slave-fljv"
server_key: "3535724607d431e779bd65abb5ce534c"
server_key_iv: "51b13149118f567159f32f95acd2e511"
server_key_version: "encryptionkey@0"
I20260619 18:47:33.415781 18778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "572ee59e5ae442b69026a7ede5834911"
format_stamp: "Formatted at 2026-06-19 18:47:33 on dist-test-slave-fljv"
server_key: "3535724607d431e779bd65abb5ce534c"
server_key_iv: "51b13149118f567159f32f95acd2e511"
server_key_version: "encryptionkey@0"
I20260619 18:47:33.419721 18778 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:47:33.422541 18793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:33.423923 18778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:33.424098 18778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "572ee59e5ae442b69026a7ede5834911"
format_stamp: "Formatted at 2026-06-19 18:47:33 on dist-test-slave-fljv"
server_key: "3535724607d431e779bd65abb5ce534c"
server_key_iv: "51b13149118f567159f32f95acd2e511"
server_key_version: "encryptionkey@0"
I20260619 18:47:33.424208 18778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:33.450660 18778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:33.454159 18778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:33.454371 18778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:33.462054 18778 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:42293
I20260619 18:47:33.462100 18845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:42293 every 8 connection(s)
I20260619 18:47:33.463161 18778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:33.466152 18846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:33.468106 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 18778
I20260619 18:47:33.468220 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:33.468473 13227 external_mini_cluster.cc:1428] Setting key 1f1f586c2dfe1bcd53974f819fe47966
I20260619 18:47:33.472955 18846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911: Bootstrap starting.
I20260619 18:47:33.475687 18846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:33.476588 18846 log.cc:824] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:33.478981 18846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911: No bootstrap required, opened a new log
Jun 19 18:47:33 dist-test-slave-fljv krb5kdc[18765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:33.482903 18846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572ee59e5ae442b69026a7ede5834911" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 42293 } }
I20260619 18:47:33.483207 18846 raft_consensus.cc:374] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:33.483270 18846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 572ee59e5ae442b69026a7ede5834911, State: Initialized, Role: FOLLOWER
I20260619 18:47:33.483747 18846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572ee59e5ae442b69026a7ede5834911" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 42293 } }
I20260619 18:47:33.483880 18846 raft_consensus.cc:388] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:33.483945 18846 raft_consensus.cc:482] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:33.484043 18846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:33.485087 18846 raft_consensus.cc:504] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572ee59e5ae442b69026a7ede5834911" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 42293 } }
I20260619 18:47:33.485481 18846 leader_election.cc:302] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 572ee59e5ae442b69026a7ede5834911; no voters:
I20260619 18:47:33.485670 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:33.470005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.12.234.254:42293)
0619 18:47:33.470475 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:33.470497 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:33.470541 (+ 44us) server_negotiation.cc:407] Connection header received
0619 18:47:33.471812 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:33.471839 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:33.472311 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:33.472717 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:33.473542 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.475050 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.475973 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.476671 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.476829 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:33.479687 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:33.479693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:33.479710 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:33.479743 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:33.482264 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:33.482784 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:33.482790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:33.482796 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:33.482946 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:33.483344 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:33.483348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:33.483350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:33.483816 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:33.484008 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:33.484414 (+ 406us) server_negotiation.cc:299] Negotiation successful
0619 18:47:33.484721 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":162,"threads_started":1}
I20260619 18:47:33.485817 18846 leader_election.cc:288] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:33.486063 18851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:33.486330 18851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [term 1 LEADER]: Becoming Leader. State: Replica: 572ee59e5ae442b69026a7ede5834911, State: Running, Role: LEADER
I20260619 18:47:33.486713 18851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572ee59e5ae442b69026a7ede5834911" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 42293 } }
I20260619 18:47:33.487095 18846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:33.488379 18852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "572ee59e5ae442b69026a7ede5834911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572ee59e5ae442b69026a7ede5834911" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 42293 } } }
I20260619 18:47:33.488507 18852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:33.488482 18853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 572ee59e5ae442b69026a7ede5834911. Latest consensus state: current_term: 1 leader_uuid: "572ee59e5ae442b69026a7ede5834911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572ee59e5ae442b69026a7ede5834911" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 42293 } } }
I20260619 18:47:33.488580 18853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:33.488839 18862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:33.491777 18862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:33.497407 18862 catalog_manager.cc:1347] Generated new cluster ID: d9caaf5fc1cc41278ab232c03468af36
I20260619 18:47:33.497506 18862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:33.526520 18862 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:33.527946 18862 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:33.541752 18862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 572ee59e5ae442b69026a7ede5834911: Generated new TSK 0
I20260619 18:47:33.542759 18862 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:33.578012 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:42293
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:33.689728 18872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:33.690023 18872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:33.690069 18872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:33.694316 18872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:33.694387 18872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:33.694450 18872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:33.699980 18872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:42293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:33.701136 18872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:33.702157 18872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:33.712177 18877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:33.712337 18878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:33.712229 18872 server_base.cc:1034] running on GCE node
W20260619 18:47:33.712234 18880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:33.712797 18872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:33.713451 18872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:33.714617 18872 hybrid_clock.cc:648] HybridClock initialized: now 1781894853714605 us; error 28 us; skew 500 ppm
Jun 19 18:47:33 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894853, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:33.717392 18872 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:33.718427 18872 webserver.cc:460] Webserver started at http://127.12.234.193:41177/ using document root <none> and password file <none>
I20260619 18:47:33.718989 18872 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:33.719058 18872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:33.719241 18872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:33.721342 18872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "7a1f91a0cdb84f82b0ac71044c10f1e6"
format_stamp: "Formatted at 2026-06-19 18:47:33 on dist-test-slave-fljv"
server_key: "95d9bb49c08a754188e84c116b4815f5"
server_key_iv: "330c9f50e4c5368f04f2646d1dc6d432"
server_key_version: "encryptionkey@0"
I20260619 18:47:33.721915 18872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "7a1f91a0cdb84f82b0ac71044c10f1e6"
format_stamp: "Formatted at 2026-06-19 18:47:33 on dist-test-slave-fljv"
server_key: "95d9bb49c08a754188e84c116b4815f5"
server_key_iv: "330c9f50e4c5368f04f2646d1dc6d432"
server_key_version: "encryptionkey@0"
I20260619 18:47:33.725718 18872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:33.728434 18887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:33.729879 18872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:33.730109 18872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "7a1f91a0cdb84f82b0ac71044c10f1e6"
format_stamp: "Formatted at 2026-06-19 18:47:33 on dist-test-slave-fljv"
server_key: "95d9bb49c08a754188e84c116b4815f5"
server_key_iv: "330c9f50e4c5368f04f2646d1dc6d432"
server_key_version: "encryptionkey@0"
I20260619 18:47:33.730355 18872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:33.755517 18872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:33.758991 18872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:33.759227 18872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:33.759841 18872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:33.760712 18872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:33.760761 18872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:33.760814 18872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:33.760843 18872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:33.769351 18872 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:40415
I20260619 18:47:33.769387 19000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:40415 every 8 connection(s)
I20260619 18:47:33.770341 18872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:33.774642 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 18872
I20260619 18:47:33.774770 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:33.775079 13227 external_mini_cluster.cc:1428] Setting key bff39163eaa05f6ba2c2663b41623fdf
Jun 19 18:47:33 dist-test-slave-fljv krb5kdc[18765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894853, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:33.785771 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:33.772368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:58687 (local address 127.12.234.254:42293)
0619 18:47:33.772569 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:33.772574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:33.773548 (+ 974us) server_negotiation.cc:407] Connection header received
0619 18:47:33.775245 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:33.775249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:33.775304 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:33.775372 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:33.776891 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.777447 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.778866 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.779219 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.779331 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:33.782044 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:33.782047 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:33.782049 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:33.782076 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:33.783492 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:33.784135 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:33.784138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:33.784140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:33.784224 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:33.784635 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:33.784637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:33.784639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:33.784830 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:33.784908 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:33.785513 (+ 605us) server_negotiation.cc:299] Negotiation successful
0619 18:47:33.785640 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 18:47:33.786720 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:33.772826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:42293 (local address 127.12.234.193:58687)
0619 18:47:33.773365 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:33.773414 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:33.774921 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:33.775591 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:33.775599 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:33.776055 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:33.776725 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:33.776747 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.777581 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.777585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:33.778743 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:33.778754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.779369 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.779372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:33.779526 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:33.780125 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:33.780142 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:33.781820 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:33.783629 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:33.783636 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:33.783658 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:33.784026 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:33.784324 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:33.784327 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:33.784329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:33.784549 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:33.784950 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:33.784956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:33.785235 (+ 279us) client_negotiation.cc:769] Sending connection context
0619 18:47:33.785453 (+ 218us) client_negotiation.cc:240] Negotiation successful
0619 18:47:33.785763 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":133,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:47:33.788316 19001 heartbeater.cc:347] Connected to a master server at 127.12.234.254:42293
I20260619 18:47:33.788594 19001 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:33.789219 19001 heartbeater.cc:511] Master 127.12.234.254:42293 requested a full tablet report, sending...
I20260619 18:47:33.790899 18810 ts_manager.cc:194] Registered new tserver with Master: 7a1f91a0cdb84f82b0ac71044c10f1e6 (127.12.234.193:40415)
I20260619 18:47:33.792042 18810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:58687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:33.803521 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:42293
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:33.913642 19006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:33.913938 19006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:33.914002 19006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:33.918263 19006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:33.918340 19006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:33.918430 19006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:33.924010 19006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:42293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:33.925350 19006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:33.926721 19006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:33.938091 19014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:33.938093 19012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:33.938197 19011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:33.938325 19006 server_base.cc:1034] running on GCE node
I20260619 18:47:33.938872 19006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:33.939576 19006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:33.940762 19006 hybrid_clock.cc:648] HybridClock initialized: now 1781894853940736 us; error 37 us; skew 500 ppm
Jun 19 18:47:33 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894853, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:33.943732 19006 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:33.944676 19006 webserver.cc:460] Webserver started at http://127.12.234.194:39527/ using document root <none> and password file <none>
I20260619 18:47:33.945212 19006 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:33.945271 19006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:33.945441 19006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:33.947609 19006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "ecf9fe9186384c55b79c1daba7c1ebdb"
format_stamp: "Formatted at 2026-06-19 18:47:33 on dist-test-slave-fljv"
server_key: "40f62c7919dad4e0b1264b256e3a742b"
server_key_iv: "eb1201d0f85a4f69dd73b0ff959b7ca8"
server_key_version: "encryptionkey@0"
I20260619 18:47:33.948158 19006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "ecf9fe9186384c55b79c1daba7c1ebdb"
format_stamp: "Formatted at 2026-06-19 18:47:33 on dist-test-slave-fljv"
server_key: "40f62c7919dad4e0b1264b256e3a742b"
server_key_iv: "eb1201d0f85a4f69dd73b0ff959b7ca8"
server_key_version: "encryptionkey@0"
I20260619 18:47:33.952005 19006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 18:47:33.954591 19021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:33.955941 19006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:33.956102 19006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "ecf9fe9186384c55b79c1daba7c1ebdb"
format_stamp: "Formatted at 2026-06-19 18:47:33 on dist-test-slave-fljv"
server_key: "40f62c7919dad4e0b1264b256e3a742b"
server_key_iv: "eb1201d0f85a4f69dd73b0ff959b7ca8"
server_key_version: "encryptionkey@0"
I20260619 18:47:33.956211 19006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:33.968958 19006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:33.971971 19006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:33.972137 19006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:33.972690 19006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:33.973500 19006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:33.973546 19006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:33.973584 19006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:33.973605 19006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:33.982208 19006 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:34685
I20260619 18:47:33.982295 19134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:34685 every 8 connection(s)
I20260619 18:47:33.983170 19006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:33.989816 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 19006
I20260619 18:47:33.989930 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:33.990269 13227 external_mini_cluster.cc:1428] Setting key 6adc065333f0feca9b0c610f44105e01
Jun 19 18:47:33 dist-test-slave-fljv krb5kdc[18765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894853, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:33.999117 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:33.985077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:47415 (local address 127.12.234.254:42293)
0619 18:47:33.985253 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:33.985257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:33.986266 (+ 1009us) server_negotiation.cc:407] Connection header received
0619 18:47:33.987721 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:33.987725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:33.987783 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:33.987897 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:33.989367 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.990138 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.991695 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.991959 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.992074 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:33.995086 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:33.995090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:33.995092 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:33.995121 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:33.996550 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:33.997288 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:33.997291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:33.997293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:33.997381 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:33.997779 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:33.997781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:33.997783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:33.997982 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:33.998081 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:33.998807 (+ 726us) server_negotiation.cc:299] Negotiation successful
0619 18:47:33.998922 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:47:34.000298 19137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:33.985538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:42293 (local address 127.12.234.194:47415)
0619 18:47:33.986031 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:33.986127 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:33.987473 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:33.988124 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:33.988135 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:33.988548 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:33.989213 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:33.989236 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.990323 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.990327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:33.991560 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:33.991571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:33.992123 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:33.992126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:33.992292 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:33.992964 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:33.992983 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:33.994829 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:33.996697 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:33.996705 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:33.996729 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:33.997160 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:33.997469 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:33.997472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:33.997474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:33.997688 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:33.998106 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:33.998113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:33.998513 (+ 400us) client_negotiation.cc:769] Sending connection context
0619 18:47:33.998754 (+ 241us) client_negotiation.cc:240] Negotiation successful
0619 18:47:33.999163 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":104,"threads_started":1}
I20260619 18:47:34.001670 19135 heartbeater.cc:347] Connected to a master server at 127.12.234.254:42293
I20260619 18:47:34.001889 19135 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:34.002414 19135 heartbeater.cc:511] Master 127.12.234.254:42293 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:47:34.003516 18810 ts_manager.cc:194] Registered new tserver with Master: ecf9fe9186384c55b79c1daba7c1ebdb (127.12.234.194:34685)
I20260619 18:47:34.004300 18810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:47415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:34.018980 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:42293
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:34.134163 19140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:34.134457 19140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:34.134516 19140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:34.138695 19140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:34.138765 19140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:34.138847 19140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:34.144304 19140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:42293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:34.145433 19140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:34.146528 19140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:34.156231 19145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:34.156284 19140 server_base.cc:1034] running on GCE node
W20260619 18:47:34.156198 19146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:34.156215 19148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:34.156847 19140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:34.157465 19140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:34.158607 19140 hybrid_clock.cc:648] HybridClock initialized: now 1781894854158578 us; error 44 us; skew 500 ppm
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:34.161350 19140 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:34.162262 19140 webserver.cc:460] Webserver started at http://127.12.234.195:44311/ using document root <none> and password file <none>
I20260619 18:47:34.162822 19140 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:34.162885 19140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:34.163093 19140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:34.165146 19140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "212315b9a244465ba67c992909d6e875"
format_stamp: "Formatted at 2026-06-19 18:47:34 on dist-test-slave-fljv"
server_key: "45212694374771222f4e17b577036bf8"
server_key_iv: "b273e6b2b81297f656f9a76c4fb52b38"
server_key_version: "encryptionkey@0"
I20260619 18:47:34.165735 19140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "212315b9a244465ba67c992909d6e875"
format_stamp: "Formatted at 2026-06-19 18:47:34 on dist-test-slave-fljv"
server_key: "45212694374771222f4e17b577036bf8"
server_key_iv: "b273e6b2b81297f656f9a76c4fb52b38"
server_key_version: "encryptionkey@0"
I20260619 18:47:34.169668 19140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:34.172261 19155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:34.173676 19140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:34.173816 19140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "212315b9a244465ba67c992909d6e875"
format_stamp: "Formatted at 2026-06-19 18:47:34 on dist-test-slave-fljv"
server_key: "45212694374771222f4e17b577036bf8"
server_key_iv: "b273e6b2b81297f656f9a76c4fb52b38"
server_key_version: "encryptionkey@0"
I20260619 18:47:34.173936 19140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:34.190126 19140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:34.193419 19140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:34.193573 19140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:34.194123 19140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:34.194988 19140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:34.195036 19140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:34.195084 19140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:34.195106 19140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:34.203636 19140 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:44953
I20260619 18:47:34.203665 19268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:44953 every 8 connection(s)
I20260619 18:47:34.204586 19140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:34.206122 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 19140
I20260619 18:47:34.206259 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:34.206570 13227 external_mini_cluster.cc:1428] Setting key 6f0b0cbe1d6d5b0805643d9f5d2941d2
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:34.218498 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:34.206481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:46297 (local address 127.12.234.254:42293)
0619 18:47:34.206635 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:34.206640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:34.207681 (+ 1041us) server_negotiation.cc:407] Connection header received
0619 18:47:34.209239 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:34.209243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:34.209297 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:34.209368 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:34.210621 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.211144 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.212267 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.212488 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.212596 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:34.214954 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:34.214957 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:34.214959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:34.214983 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:34.216413 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:34.216940 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:34.216942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:34.216944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:34.217032 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:34.217347 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:34.217350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:34.217351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:34.217548 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:34.217634 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:34.218145 (+ 511us) server_negotiation.cc:299] Negotiation successful
0619 18:47:34.218328 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:47:34.219297 19271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:34.206997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:42293 (local address 127.12.234.195:46297)
0619 18:47:34.207531 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:34.207593 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:34.209029 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:34.209544 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:34.209551 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:34.209913 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:34.210483 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:34.210497 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.211262 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.211265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:34.212157 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:34.212165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.212623 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.212627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:34.212789 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:34.213344 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:34.213367 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:34.214786 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:34.216548 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:34.216554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:34.216572 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:34.216841 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:34.217115 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:34.217117 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:34.217119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:34.217261 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:34.217646 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:34.217653 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:34.217910 (+ 257us) client_negotiation.cc:769] Sending connection context
0619 18:47:34.218086 (+ 176us) client_negotiation.cc:240] Negotiation successful
0619 18:47:34.218464 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":126,"threads_started":1}
I20260619 18:47:34.220366 19269 heartbeater.cc:347] Connected to a master server at 127.12.234.254:42293
I20260619 18:47:34.220623 19269 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:34.221097 19269 heartbeater.cc:511] Master 127.12.234.254:42293 requested a full tablet report, sending...
I20260619 18:47:34.222055 18810 ts_manager.cc:194] Registered new tserver with Master: 212315b9a244465ba67c992909d6e875 (127.12.234.195:44953)
I20260619 18:47:34.222647 18810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:46297
I20260619 18:47:34.232349 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/test-admin.keytab.
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:34.247946 13227 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 18:47:34.250810 19276 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:34.249331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.12.234.193:40415)
0619 18:47:34.249630 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:34.249647 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:34.249674 (+ 27us) server_negotiation.cc:407] Connection header received
0619 18:47:34.249748 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:34.249754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:34.249958 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:34.250110 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:34.250650 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46030: BlockingRecv error: recv got EOF from 127.0.0.1:46030 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":63,"threads_started":1}
W20260619 18:47:34.250958 13227 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5cc' of type FILE
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:34.252090 13227 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:34.253027 13227 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:34.263397 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:34.253933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.12.234.193:40415)
0619 18:47:34.254080 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:34.254085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:34.254248 (+ 163us) server_negotiation.cc:407] Connection header received
0619 18:47:34.254348 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:34.254350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:34.254394 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:34.254469 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:34.254990 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.255808 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.256633 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.257028 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.257128 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:34.259555 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:34.259592 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:34.259611 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:34.259644 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:34.261343 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:34.261801 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:34.261807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:34.261809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:34.261905 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:34.262236 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:34.262240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:34.262242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:34.262549 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:34.262724 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:34.263014 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 18:47:34.263245 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260619 18:47:34.267589 19276 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:34.266535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.12.234.193:40415)
0619 18:47:34.266679 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:34.266683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:34.266759 (+ 76us) server_negotiation.cc:407] Connection header received
0619 18:47:34.266865 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:34.266867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:34.266905 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:34.266971 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:34.267473 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46036: BlockingRecv error: recv got EOF from 127.0.0.1:46036 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 18:47:34.267891 13227 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5cc)
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:34.268889 13227 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:34.269740 13227 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:34.278831 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:34.270631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.12.234.193:40415)
0619 18:47:34.270762 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:34.270766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:34.270838 (+ 72us) server_negotiation.cc:407] Connection header received
0619 18:47:34.270965 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:34.270976 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:34.271020 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:34.271082 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:34.271584 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.272125 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.272891 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.273135 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.273222 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:34.275569 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:34.275573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:34.275575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:34.275605 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:34.277126 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:34.277592 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:34.277595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:34.277596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:34.277686 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:34.277960 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:34.277964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:34.277966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:34.278141 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:34.278284 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:34.278533 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 18:47:34.278649 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 18:47:34.281754 19276 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:34.280710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.12.234.193:40415)
0619 18:47:34.280850 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:34.280854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:34.280919 (+ 65us) server_negotiation.cc:407] Connection header received
0619 18:47:34.281056 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:34.281060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:34.281103 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:34.281168 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:34.281649 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46056: BlockingRecv error: recv got EOF from 127.0.0.1:46056 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 18:47:34.281872 13227 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781894826428600-13227-0/krb5kdc/krb5cc' of type FILE
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:34.282891 13227 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:34.283746 13227 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[18765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:47:34.292548 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:34.284648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.12.234.193:40415)
0619 18:47:34.284819 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:34.284824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:34.284839 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:34.284939 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:34.284942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:34.284982 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:34.285114 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:34.285668 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.286168 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.287058 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:34.287326 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:34.287409 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:34.289727 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:34.289731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:34.289733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:34.289760 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:34.291091 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:34.291477 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:34.291480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:34.291481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:34.291567 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:34.291808 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:34.291811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:34.291812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:34.292019 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:34.292127 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:34.292289 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 18:47:34.292386 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 18:47:34.293679 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 18872
I20260619 18:47:34.298635 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 19006
I20260619 18:47:34.303254 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 19140
I20260619 18:47:34.307878 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 18778
2026-06-19T18:47:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2119 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[19289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[19289](info): set up 2 sockets
Jun 19 18:47:34 dist-test-slave-fljv krb5kdc[19289](info): commencing operation
krb5kdc: starting...
W20260619 18:47:35.274504 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:35 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.254.keytab.
Entry for principal oryx/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.254.keytab.
I20260619 18:47:35.394982 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:44503
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.254.keytab
--principal=oryx/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:44503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:35.509359 19302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:35.509657 19302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:35.509716 19302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:35.513804 19302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:35.513875 19302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:35.513902 19302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:35.513923 19302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:35.513949 19302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:35.519399 19302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:44503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.254.keytab
--principal=oryx/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:44503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:35.520561 19302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:35.521610 19302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:35.530543 19308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:35.530520 19307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:35.530568 19310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:35.530529 19302 server_base.cc:1034] running on GCE node
I20260619 18:47:35.531399 19302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:35.532594 19302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:35.533788 19302 hybrid_clock.cc:648] HybridClock initialized: now 1781894855533765 us; error 39 us; skew 500 ppm
Jun 19 18:47:35 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894855, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:35.536717 19302 init.cc:377] Logged in from keytab as oryx/127.12.234.254@KRBTEST.COM (short username oryx)
I20260619 18:47:35.537649 19302 webserver.cc:460] Webserver started at http://127.12.234.254:35833/ using document root <none> and password file <none>
I20260619 18:47:35.538287 19302 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:35.538360 19302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:35.538569 19302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:35.540899 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "bfa9f9d9741744ca8c9985b264c1d108"
format_stamp: "Formatted at 2026-06-19 18:47:35 on dist-test-slave-fljv"
server_key: "40096dd5717cee3bc1a913da6864fa0e"
server_key_iv: "a33ecb0e2e60b6dc24b0e56241cf140d"
server_key_version: "encryptionkey@0"
I20260619 18:47:35.541539 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "bfa9f9d9741744ca8c9985b264c1d108"
format_stamp: "Formatted at 2026-06-19 18:47:35 on dist-test-slave-fljv"
server_key: "40096dd5717cee3bc1a913da6864fa0e"
server_key_iv: "a33ecb0e2e60b6dc24b0e56241cf140d"
server_key_version: "encryptionkey@0"
I20260619 18:47:35.545523 19302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:35.548159 19317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:35.549535 19302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:35.549657 19302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "bfa9f9d9741744ca8c9985b264c1d108"
format_stamp: "Formatted at 2026-06-19 18:47:35 on dist-test-slave-fljv"
server_key: "40096dd5717cee3bc1a913da6864fa0e"
server_key_iv: "a33ecb0e2e60b6dc24b0e56241cf140d"
server_key_version: "encryptionkey@0"
I20260619 18:47:35.549768 19302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:35.565739 19302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:35.569041 19302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:35.569239 19302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:35.576171 19302 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:44503
I20260619 18:47:35.576427 19369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:44503 every 8 connection(s)
I20260619 18:47:35.577180 19302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:35.580024 19370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:35.581204 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 19302
I20260619 18:47:35.581336 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:35.581598 13227 external_mini_cluster.cc:1428] Setting key 6a2347ff5b56c411eb8339f0424ed024
I20260619 18:47:35.586727 19370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: Bootstrap starting.
I20260619 18:47:35.589358 19370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:35.590090 19370 log.cc:824] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: Log is configured to *not* fsync() on all Append() calls
Jun 19 18:47:35 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:35.592242 19370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: No bootstrap required, opened a new log
I20260619 18:47:35.596669 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:35.583023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.12.234.254:44503)
0619 18:47:35.583449 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:35.583469 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:35.583513 (+ 44us) server_negotiation.cc:407] Connection header received
0619 18:47:35.584309 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:35.584334 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:35.584765 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:35.585162 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:35.585793 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:35.587243 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:35.588198 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:35.588882 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:35.589050 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:35.591515 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:35.591521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:35.591540 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:35.591572 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:35.593507 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:35.593961 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:35.593966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:35.593972 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:35.594124 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:35.594435 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:35.594438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:35.594440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:35.594965 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:35.595157 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:35.595363 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 18:47:35.595672 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":119,"threads_started":1}
I20260619 18:47:35.596669 19370 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } }
I20260619 18:47:35.596979 19370 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:35.597036 19370 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa9f9d9741744ca8c9985b264c1d108, State: Initialized, Role: FOLLOWER
I20260619 18:47:35.597517 19370 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } }
I20260619 18:47:35.597667 19370 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:35.597729 19370 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:35.597821 19370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:35.598814 19370 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } }
I20260619 18:47:35.599176 19370 leader_election.cc:302] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfa9f9d9741744ca8c9985b264c1d108; no voters:
I20260619 18:47:35.599506 19370 leader_election.cc:288] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:35.599598 19375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:35.599808 19375 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 1 LEADER]: Becoming Leader. State: Replica: bfa9f9d9741744ca8c9985b264c1d108, State: Running, Role: LEADER
I20260619 18:47:35.600127 19375 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } }
I20260619 18:47:35.600643 19370 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:35.601706 19376 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } }
I20260619 18:47:35.601698 19377 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa9f9d9741744ca8c9985b264c1d108. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } }
I20260619 18:47:35.601837 19376 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:35.601837 19377 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:35.602254 19384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:35.605506 19384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:35.611343 19384 catalog_manager.cc:1347] Generated new cluster ID: dff4f39ee9de43c7843284c70fc51efe
I20260619 18:47:35.611424 19384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:35.618964 19384 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:35.620100 19384 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:35.624347 19384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: Generated new TSK 0
I20260619 18:47:35.625243 19384 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab.
Entry for principal oryx/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab.
I20260619 18:47:35.659265 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44503
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab
--principal=oryx/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:35.771525 19396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:35.771799 19396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:35.771847 19396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:35.775816 19396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:35.775874 19396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:35.775931 19396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:35.781359 19396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab
--principal=oryx/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:35.782402 19396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:35.783380 19396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:35.792973 19401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:35.793043 19402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:35.792975 19404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:35.793112 19396 server_base.cc:1034] running on GCE node
I20260619 18:47:35.793645 19396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:35.794333 19396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:35.795507 19396 hybrid_clock.cc:648] HybridClock initialized: now 1781894855795493 us; error 31 us; skew 500 ppm
Jun 19 18:47:35 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894855, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:35.798142 19396 init.cc:377] Logged in from keytab as oryx/127.12.234.193@KRBTEST.COM (short username oryx)
I20260619 18:47:35.799077 19396 webserver.cc:460] Webserver started at http://127.12.234.193:45409/ using document root <none> and password file <none>
I20260619 18:47:35.799616 19396 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:35.799676 19396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:35.799845 19396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:35.801996 19396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "e175e83617ab4b298c7abd17b946b6d2"
format_stamp: "Formatted at 2026-06-19 18:47:35 on dist-test-slave-fljv"
server_key: "c6d1d235bd1782e1d112dbcf400689c6"
server_key_iv: "2cf64028a8310876a4c8f4de23a34519"
server_key_version: "encryptionkey@0"
I20260619 18:47:35.802570 19396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "e175e83617ab4b298c7abd17b946b6d2"
format_stamp: "Formatted at 2026-06-19 18:47:35 on dist-test-slave-fljv"
server_key: "c6d1d235bd1782e1d112dbcf400689c6"
server_key_iv: "2cf64028a8310876a4c8f4de23a34519"
server_key_version: "encryptionkey@0"
I20260619 18:47:35.806161 19396 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:35.808638 19411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:35.809934 19396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 18:47:35.810041 19396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "e175e83617ab4b298c7abd17b946b6d2"
format_stamp: "Formatted at 2026-06-19 18:47:35 on dist-test-slave-fljv"
server_key: "c6d1d235bd1782e1d112dbcf400689c6"
server_key_iv: "2cf64028a8310876a4c8f4de23a34519"
server_key_version: "encryptionkey@0"
I20260619 18:47:35.810163 19396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:35.835633 19396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:35.838827 19396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:35.839000 19396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:35.839581 19396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:35.840407 19396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:35.840453 19396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:35.840504 19396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:35.840533 19396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:35.848861 19396 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:42443
I20260619 18:47:35.848925 19524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:42443 every 8 connection(s)
I20260619 18:47:35.849851 19396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:35.855626 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 19396
I20260619 18:47:35.855751 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:35.856020 13227 external_mini_cluster.cc:1428] Setting key ecfbf81f973da8cbfb38f1e56a2ca3ec
Jun 19 18:47:35 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894855, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:35.863840 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:35.851561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:57251 (local address 127.12.234.254:44503)
0619 18:47:35.851676 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:35.851680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:35.852591 (+ 911us) server_negotiation.cc:407] Connection header received
0619 18:47:35.854019 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:35.854023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:35.854082 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:35.854158 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:35.855403 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:35.856012 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:35.857152 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:35.857379 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:35.857468 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:35.860028 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:35.860032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:35.860035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:35.860063 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:35.861582 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:35.862152 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:35.862155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:35.862157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:35.862268 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:35.862656 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:35.862658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:35.862660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:35.862842 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:35.862914 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:35.863594 (+ 680us) server_negotiation.cc:299] Negotiation successful
0619 18:47:35.863716 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 18:47:35.865010 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:35.851919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:57251)
0619 18:47:35.852402 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:35.852448 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:35.853774 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:35.854372 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:35.854379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:35.854761 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:35.855242 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:35.855255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:35.856140 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:35.856142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:35.857029 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:35.857038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:35.857525 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:35.857529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:35.857692 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:35.858277 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:35.858293 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:35.859836 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:35.861691 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:35.861696 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:35.861710 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:35.862050 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:35.862351 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:35.862354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:35.862356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:35.862557 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:35.862937 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:35.862944 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:35.863287 (+ 343us) client_negotiation.cc:769] Sending connection context
0619 18:47:35.863540 (+ 253us) client_negotiation.cc:240] Negotiation successful
0619 18:47:35.863906 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for oryx/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:35.866253 19525 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44503
I20260619 18:47:35.866479 19525 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:35.867022 19525 heartbeater.cc:511] Master 127.12.234.254:44503 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.194@KRBTEST.COM" created.
I20260619 18:47:35.868599 19334 ts_manager.cc:194] Registered new tserver with Master: e175e83617ab4b298c7abd17b946b6d2 (127.12.234.193:42443)
I20260619 18:47:35.870224 19334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:57251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab.
Entry for principal oryx/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab.
I20260619 18:47:35.885286 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44503
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab
--principal=oryx/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:35.997934 19530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:35.998247 19530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:35.998291 19530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:36.002579 19530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:36.002657 19530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:36.002732 19530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:36.008131 19530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab
--principal=oryx/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:36.009194 19530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:36.010260 19530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:36.019834 19535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.019829 19536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.019845 19538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:36.020224 19530 server_base.cc:1034] running on GCE node
I20260619 18:47:36.020742 19530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:36.021464 19530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:36.022640 19530 hybrid_clock.cc:648] HybridClock initialized: now 1781894856022611 us; error 42 us; skew 500 ppm
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:36.025372 19530 init.cc:377] Logged in from keytab as oryx/127.12.234.194@KRBTEST.COM (short username oryx)
I20260619 18:47:36.026309 19530 webserver.cc:460] Webserver started at http://127.12.234.194:35369/ using document root <none> and password file <none>
I20260619 18:47:36.026865 19530 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:36.026925 19530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:36.027124 19530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:36.029187 19530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "663ae2a2502b7f94cbb0e37f96a4979e"
server_key_iv: "1736821f5a3ae626757cf10746495b8b"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.029714 19530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "663ae2a2502b7f94cbb0e37f96a4979e"
server_key_iv: "1736821f5a3ae626757cf10746495b8b"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.033582 19530 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:36.036063 19545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.037340 19530 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:36.037459 19530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "663ae2a2502b7f94cbb0e37f96a4979e"
server_key_iv: "1736821f5a3ae626757cf10746495b8b"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.037569 19530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:36.059811 19530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:36.062850 19530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:36.063040 19530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:36.063722 19530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:36.064590 19530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:36.064637 19530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.064679 19530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:36.064707 19530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.073055 19530 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:38539
I20260619 18:47:36.073076 19658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:38539 every 8 connection(s)
I20260619 18:47:36.073997 19530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:36.081907 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 19530
I20260619 18:47:36.082119 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:36.082485 13227 external_mini_cluster.cc:1428] Setting key 4c10c8887a0155bee19ac955bc8ebdb4
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:36.087598 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.075840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:43709 (local address 127.12.234.254:44503)
0619 18:47:36.075989 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.075993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.076860 (+ 867us) server_negotiation.cc:407] Connection header received
0619 18:47:36.078201 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.078205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.078263 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.078340 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.079562 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.080104 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.081241 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.081457 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.081547 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.084180 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.084183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.084185 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.084210 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.085538 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.086033 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.086037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.086039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.086130 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.086460 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.086463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.086465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.086689 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.086774 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.087295 (+ 521us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.087426 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 18:47:36.088449 19661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.076227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.194:43709)
0619 18:47:36.076705 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.076750 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.077967 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.078511 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.078544 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.078914 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.079404 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.079418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.080210 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.080213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.081122 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.081132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.081609 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.081612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.081809 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.082477 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.082497 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.084003 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.085653 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.085660 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.085677 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.085936 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.086225 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.086229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.086231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.086375 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.086791 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.086798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.087067 (+ 269us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.087239 (+ 172us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.087579 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":114,"threads_started":1}
I20260619 18:47:36.089617 19659 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44503
I20260619 18:47:36.089915 19659 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:36.090581 19659 heartbeater.cc:511] Master 127.12.234.254:44503 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:47:36.091832 19334 ts_manager.cc:194] Registered new tserver with Master: 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539)
I20260619 18:47:36.092420 19334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:43709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab.
Entry for principal oryx/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab.
I20260619 18:47:36.109007 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44503
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab
--principal=oryx/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:36.219616 19664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:36.219890 19664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:36.219933 19664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:36.224014 19664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:36.224089 19664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:36.224155 19664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:36.229558 19664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab
--principal=oryx/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:36.230691 19664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:36.231711 19664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:36.241025 19672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.241029 19669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.241039 19670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:36.241346 19664 server_base.cc:1034] running on GCE node
I20260619 18:47:36.241847 19664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:36.242600 19664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:36.243772 19664 hybrid_clock.cc:648] HybridClock initialized: now 1781894856243753 us; error 29 us; skew 500 ppm
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:36.246634 19664 init.cc:377] Logged in from keytab as oryx/127.12.234.195@KRBTEST.COM (short username oryx)
I20260619 18:47:36.247555 19664 webserver.cc:460] Webserver started at http://127.12.234.195:34305/ using document root <none> and password file <none>
I20260619 18:47:36.248107 19664 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:36.248170 19664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:36.248338 19664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:36.250382 19664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "3ff50c2d32a5412d8af73c53de93d3cf"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "25b835f3c01d74f33f7ded7035453c1a"
server_key_iv: "bcc40b09b4bfe6ed65d6bd198c745067"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.250892 19664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "3ff50c2d32a5412d8af73c53de93d3cf"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "25b835f3c01d74f33f7ded7035453c1a"
server_key_iv: "bcc40b09b4bfe6ed65d6bd198c745067"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.254776 19664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:47:36.257139 19679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.258405 19664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:36.258509 19664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "3ff50c2d32a5412d8af73c53de93d3cf"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "25b835f3c01d74f33f7ded7035453c1a"
server_key_iv: "bcc40b09b4bfe6ed65d6bd198c745067"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.258615 19664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:36.279043 19664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:36.282121 19664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:36.282337 19664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:36.282963 19664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:36.283905 19664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:36.283954 19664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.283993 19664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:36.284006 19664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.292832 19664 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:42317
I20260619 18:47:36.292855 19792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:42317 every 8 connection(s)
I20260619 18:47:36.293772 19664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:36.295238 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 19664
I20260619 18:47:36.295356 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:36.295697 13227 external_mini_cluster.cc:1428] Setting key 0f921fd9ea375ed91557c75a1f6f1630
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:36.307301 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.295663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:34217 (local address 127.12.234.254:44503)
0619 18:47:36.295789 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.295793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.296893 (+ 1100us) server_negotiation.cc:407] Connection header received
0619 18:47:36.298316 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.298319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.298379 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.298451 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.299628 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.300235 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.301411 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.301633 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.301741 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.304122 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.304125 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.304127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.304153 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.305307 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.305797 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.305801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.305802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.305883 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.306207 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.306211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.306212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.306394 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.306473 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.307056 (+ 583us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.307174 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:47:36.308133 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.296196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.195:34217)
0619 18:47:36.296741 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.296795 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.298088 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.298630 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.298636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.299007 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.299492 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.299505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.300335 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.300337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.301306 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.301317 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.301781 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.301784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.301928 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.302493 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.302509 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.303945 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.305405 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.305410 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.305424 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.305696 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.305977 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.305979 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.305980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.306109 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.306476 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.306481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.306757 (+ 276us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.306921 (+ 164us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.307187 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":143,"threads_started":1}
I20260619 18:47:36.309362 19793 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44503
I20260619 18:47:36.309620 19793 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:36.310146 19793 heartbeater.cc:511] Master 127.12.234.254:44503 requested a full tablet report, sending...
I20260619 18:47:36.311205 19334 ts_manager.cc:194] Registered new tserver with Master: 3ff50c2d32a5412d8af73c53de93d3cf (127.12.234.195:42317)
I20260619 18:47:36.311760 19334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:34217
I20260619 18:47:36.321610 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.234.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.253.keytab.
Entry for principal oryx/127.12.234.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.253.keytab.
I20260619 18:47:36.347966 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.253:36807
--webserver_interface=127.12.234.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.253.keytab
--principal=oryx/127.12.234.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:44503,127.12.234.253:36807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:36.458352 19798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:36.458623 19798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:36.458662 19798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:36.462738 19798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:36.462807 19798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:36.462826 19798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:36.462841 19798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:36.462857 19798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:36.468254 19798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:44503,127.12.234.253:36807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.253.keytab
--principal=oryx/127.12.234.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.253:36807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.234.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
W20260619 18:47:36.468484 19798 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.12.234.254:44503,127.12.234.253:36807'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 18:47:36.469388 19798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:36.470451 19798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:36.478644 19806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.478627 19804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.478631 19803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:36.479084 19798 server_base.cc:1034] running on GCE node
I20260619 18:47:36.479615 19798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:36.480749 19798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:36.481902 19798 hybrid_clock.cc:648] HybridClock initialized: now 1781894856481883 us; error 32 us; skew 500 ppm
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:36.484743 19798 init.cc:377] Logged in from keytab as oryx/127.12.234.253@KRBTEST.COM (short username oryx)
I20260619 18:47:36.485643 19798 webserver.cc:460] Webserver started at http://127.12.234.253:40297/ using document root <none> and password file <none>
I20260619 18:47:36.486203 19798 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:36.486275 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:36.486457 19798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:36.488605 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data/instance:
uuid: "c675f9ddf30346499d349179d2668d8a"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "3846c1dfbafadcad2a8527b66aa47d3e"
server_key_iv: "376f072985437faaf17d4ce916d037c1"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.489166 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal/instance:
uuid: "c675f9ddf30346499d349179d2668d8a"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "3846c1dfbafadcad2a8527b66aa47d3e"
server_key_iv: "376f072985437faaf17d4ce916d037c1"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.492969 19798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:36.495566 19813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.496863 19798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 18:47:36.496968 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal
uuid: "c675f9ddf30346499d349179d2668d8a"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "3846c1dfbafadcad2a8527b66aa47d3e"
server_key_iv: "376f072985437faaf17d4ce916d037c1"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.497110 19798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:36.521895 19798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:36.525151 19798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:36.525319 19798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:36.532068 19798 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.253:36807
I20260619 18:47:36.532312 19865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.253:36807 every 8 connection(s)
I20260619 18:47:36.533053 19798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
I20260619 18:47:36.534288 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 19798
I20260619 18:47:36.536335 19866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:36.540309 19866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.253@KRBTEST.COM
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.253@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:36.549115 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.535860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48632 (local address 127.12.234.253:36807)
0619 18:47:36.536288 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.536305 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.536341 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:47:36.537013 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.537031 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.537372 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.537701 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.538645 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.539663 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.540518 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.541056 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.541202 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.544050 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.544057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.544068 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.544101 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.545908 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.546430 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.546435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.546440 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.546586 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.546868 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.546870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.546871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.547354 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.547597 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.547828 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.548161 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":136,"threads_started":1}
I20260619 18:47:36.551573 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.542092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33154 (local address 127.12.234.254:44503)
0619 18:47:36.542259 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.542263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.542692 (+ 429us) server_negotiation.cc:407] Connection header received
0619 18:47:36.542903 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.542907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.542961 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.543054 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.543745 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.544483 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.545445 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.545772 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.545887 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.548313 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.548316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.548318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.548345 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.549703 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.550215 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.550219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.550220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.550314 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.550562 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.550565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.550566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.550857 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.550951 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.551298 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.551422 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 18:47:36.551641 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.542050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:33154)
0619 18:47:36.542536 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.542602 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.542739 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.543121 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.543127 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.543454 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.543624 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.543633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.544661 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.544665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.545327 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.545336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.545879 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.545881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.545967 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.546487 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.546499 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.548164 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.549800 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.549806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.549812 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.550086 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.550402 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.550404 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.550406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.550482 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.550953 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.550958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.551114 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.551238 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.551438 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":171,"threads_started":1}
I20260619 18:47:36.552737 19866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.253@KRBTEST.COM for oryx/127.12.234.253@KRBTEST.COM
I20260619 18:47:36.560390 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.553187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.0.0.1:48634)
0619 18:47:36.553288 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.553302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.553373 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.553596 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.553599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.553741 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.553820 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.553825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.554877 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.554880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.555434 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.555442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.555838 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.555841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.555952 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.556384 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.556395 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.557587 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.559145 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.559147 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.559149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.559325 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.559580 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.559582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.559583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.559671 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.559955 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.559957 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.560047 (+ 90us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.560123 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.560232 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":3}
I20260619 18:47:36.560411 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.553259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.12.234.253:36807)
0619 18:47:36.553351 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.553355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.553385 (+ 30us) server_negotiation.cc:407] Connection header received
0619 18:47:36.553445 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.553448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.553505 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.553562 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.553911 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.554769 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.555538 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.555750 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.555833 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.557733 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.557736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.557738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.557760 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.559038 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.559432 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.559435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.559436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.559510 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.559732 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.559733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.559734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.559894 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.559961 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.560164 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.560284 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":39}
I20260619 18:47:36.563437 19866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Bootstrap starting.
I20260619 18:47:36.565682 19866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:36.566442 19866 log.cc:824] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:36.568356 19866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: No bootstrap required, opened a new log
I20260619 18:47:36.571424 19866 raft_consensus.cc:348] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } }
I20260619 18:47:36.571643 19866 raft_consensus.cc:374] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:36.571693 19866 raft_consensus.cc:729] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c675f9ddf30346499d349179d2668d8a, State: Initialized, Role: FOLLOWER
I20260619 18:47:36.572188 19866 consensus_queue.cc:260] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } }
I20260619 18:47:36.572667 19872 sys_catalog.cc:455] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } } }
I20260619 18:47:36.572806 19872 sys_catalog.cc:458] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:47:36.573410 19866 sys_catalog.cc:564] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: configured and running, proceeding with master startup.
W20260619 18:47:36.578148 19883 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:47:36.578366 19883 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:47:36.580085 19798 master_runner.cc:428] Detected that this master c675f9ddf30346499d349179d2668d8a is joining an existing cluster
I20260619 18:47:36.580137 19798 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.234.253:36807
I20260619 18:47:36.591656 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.582095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.12.234.254:44503)
0619 18:47:36.582236 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.582241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.582723 (+ 482us) server_negotiation.cc:407] Connection header received
0619 18:47:36.582886 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.582889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.582931 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.582976 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.583769 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.584643 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.585835 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.586219 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.586281 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.587932 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.587937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.587940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.587976 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.589871 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.590350 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.590354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.590356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.590440 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.590718 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.590721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.590723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.590955 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.591039 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.591372 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.591504 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:47:36.591650 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.582325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:33160)
0619 18:47:36.582645 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.582659 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.582789 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.583101 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.583103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.583412 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.583647 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.583665 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.584759 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.584761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.585713 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.585723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.586400 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.586403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.586532 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.587239 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.587255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.587770 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.589980 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.589983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.589985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.590241 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.590550 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.590552 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.590555 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.590625 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.591101 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.591104 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.591191 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.591299 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.591439 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":129,"spinlock_wait_cycles":27264,"thread_start_us":91,"threads_started":1}
I20260619 18:47:36.591668 19890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.582075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.0.0.1:48640)
0619 18:47:36.582647 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.582659 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.582788 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.583096 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.583099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.583413 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.583643 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.583653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.584706 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.584709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.585713 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.585723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.586396 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.586399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.586528 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.587586 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.587599 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.587993 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.589908 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.589912 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.589914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.590395 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.590654 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.590656 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.590658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.590725 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.591080 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.591083 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.591192 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.591292 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.591524 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":221,"spinlock_wait_cycles":11136,"thread_start_us":110,"threads_started":1}
I20260619 18:47:36.592408 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.582435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.12.234.253:36807)
0619 18:47:36.582527 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.582531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.582719 (+ 188us) server_negotiation.cc:407] Connection header received
0619 18:47:36.582878 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.582881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.582917 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.582961 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.583757 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.584575 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.585825 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.586137 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.586242 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.588107 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.588110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.588113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.588143 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.589628 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.590479 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.590482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.590484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.590565 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.590801 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.590804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.590806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.590975 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.591040 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.592148 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.592264 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894855, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.254@KRBTEST.COM for oryx/127.12.234.253@KRBTEST.COM
I20260619 18:47:36.605569 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.596129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.0.0.1:48642)
0619 18:47:36.596455 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.596506 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.596631 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.596924 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.596929 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.597332 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.597435 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.597441 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.598440 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.598443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.599152 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.599162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.599712 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.599715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.599803 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.600378 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.600389 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.601979 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.603800 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.603809 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.603815 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.604132 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.604435 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.604439 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.604441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.604550 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.604952 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.604959 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.605079 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.605206 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.605402 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260619 18:47:36.605559 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.596147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48642 (local address 127.12.234.253:36807)
0619 18:47:36.596274 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.596278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.596576 (+ 298us) server_negotiation.cc:407] Connection header received
0619 18:47:36.596753 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.596756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.596807 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.596886 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.597534 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.598321 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.599260 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.599593 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.599693 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.602088 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.602092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.602094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.602118 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.603685 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.604237 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.604240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.604242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.604327 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.604637 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.604640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.604642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.604850 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.604936 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.605274 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.605391 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 18:47:36.607127 19334 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.234.253:36807: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c675f9ddf30346499d349179d2668d8a"
member_type: NON_VOTER
last_known_addr {
host: "127.12.234.253"
port: 36807
}
attrs {
promote: true
}
}
dest_uuid: "bfa9f9d9741744ca8c9985b264c1d108"
cas_config_opid_index: -1
I20260619 18:47:36.607858 19334 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } }
I20260619 18:47:36.609469 19894 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c675f9ddf30346499d349179d2668d8a (127.12.234.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } } }
I20260619 18:47:36.610380 19840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:36.611270 19894 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.234.253:36807
I20260619 18:47:36.611745 19798 master.cc:537] Master@127.12.234.253:36807 shutting down...
I20260619 18:47:36.612195 19840 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Refusing update from remote peer bfa9f9d9741744ca8c9985b264c1d108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 18:47:36.613039 19894 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:36.613682 19893 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } } }
I20260619 18:47:36.613867 19893 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
WARNING: no policy specified for oryx/127.12.234.252@KRBTEST.COM; defaulting to no policy
W20260619 18:47:36.615504 19318 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260619 18:47:36.616192 19798 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 18:47:36.616356 19318 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 -> Peer c675f9ddf30346499d349179d2668d8a (127.12.234.253:36807): Couldn't send request to peer c675f9ddf30346499d349179d2668d8a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 18:47:36.616482 19798 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:36.616585 19798 tablet_replica.cc:330] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.252@KRBTEST.COM" created.
I20260619 18:47:36.632279 19798 master.cc:552] Master@127.12.234.253:36807 shutdown complete.
I20260619 18:47:36.632345 19798 master_runner.cc:305] Clearing existing system tablet
I20260619 18:47:36.632661 19798 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:36.632725 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.252.keytab.
Entry for principal oryx/127.12.234.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.252.keytab.
I20260619 18:47:36.633592 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.252:37131
--webserver_interface=127.12.234.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.252.keytab
--principal=oryx/127.12.234.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:44503,127.12.234.253:36807,127.12.234.252:37131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:47:36.634339 19798 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:36.635246 19901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.635468 19798 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:36.635669 19798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:36.635733 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal
uuid: "c675f9ddf30346499d349179d2668d8a"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "3846c1dfbafadcad2a8527b66aa47d3e"
server_key_iv: "376f072985437faaf17d4ce916d037c1"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.637593 19798 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:36.640200 19798 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 18:47:36.640345 19798 log.cc:1196] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 18:47:36.640643 19798 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Deleting consensus metadata
I20260619 18:47:36.640849 19798 master_runner.cc:315] Copying system tablet from 127.12.234.254:44503
I20260619 18:47:36.642165 19798 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: tablet copy: Beginning tablet copy session from remote peer at address 127.12.234.254:44503
I20260619 18:47:36.651837 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.643043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.12.234.254:44503)
0619 18:47:36.643181 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.643186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.643447 (+ 261us) server_negotiation.cc:407] Connection header received
0619 18:47:36.643676 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.643680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.643738 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.643795 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.644472 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.645293 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.646546 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.646916 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.646999 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.648474 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.648478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.648481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.648509 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.650073 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.650575 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.650578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.650580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.650675 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.650946 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.650949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.650950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.651154 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.651219 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.651536 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.651670 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:47:36.652498 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.643063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:33168)
0619 18:47:36.643344 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.643359 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.643539 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.643956 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.643959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.644182 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.644326 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.644334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.645448 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.645451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.646401 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.646413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.647187 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.647191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.647291 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.647875 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.647888 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.648316 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.650218 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.650222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.650224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.650472 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.650776 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.650778 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.650780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.650860 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.651283 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.651286 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.651379 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.652163 (+ 784us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.652311 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":96,"threads_started":1}
I20260619 18:47:36.653156 19354 tablet_copy_service.cc:140] P bfa9f9d9741744ca8c9985b264c1d108: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c675f9ddf30346499d349179d2668d8a ({username='oryx', principal='oryx/127.12.234.253@KRBTEST.COM'} at 127.0.0.1:33168)
I20260619 18:47:36.653350 19354 tablet_copy_service.cc:161] P bfa9f9d9741744ca8c9985b264c1d108: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c675f9ddf30346499d349179d2668d8a at {username='oryx', principal='oryx/127.12.234.253@KRBTEST.COM'} at 127.0.0.1:33168: session id = c675f9ddf30346499d349179d2668d8a-00000000000000000000000000000000
I20260619 18:47:36.655252 19354 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 18:47:36.657122 19798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 18:47:36.659515 19798 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: tablet copy: Starting download of 0 data blocks...
I20260619 18:47:36.659715 19798 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: tablet copy: Starting download of 1 WAL segments...
I20260619 18:47:36.661703 19798 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 18:47:36.663554 19354 tablet_copy_service.cc:342] P bfa9f9d9741744ca8c9985b264c1d108: Request end of tablet copy session c675f9ddf30346499d349179d2668d8a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.234.253@KRBTEST.COM'} at 127.0.0.1:33168
I20260619 18:47:36.663697 19354 tablet_copy_service.cc:433] P bfa9f9d9741744ca8c9985b264c1d108: ending tablet copy session c675f9ddf30346499d349179d2668d8a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c675f9ddf30346499d349179d2668d8a
I20260619 18:47:36.670554 19798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:36.672295 19908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.675257 19909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.675323 19911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:36.678421 19798 server_base.cc:1034] running on GCE node
I20260619 18:47:36.678642 19798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:36.678972 19798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:36.680099 19798 hybrid_clock.cc:648] HybridClock initialized: now 1781894856680077 us; error 42 us; skew 500 ppm
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:36.681689 19798 init.cc:377] Logged in from keytab as oryx/127.12.234.253@KRBTEST.COM (short username oryx)
I20260619 18:47:36.682202 19798 webserver.cc:460] Webserver started at http://127.12.234.253:40911/ using document root <none> and password file <none>
I20260619 18:47:36.682400 19798 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:36.682430 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:36.683773 19798 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:36.684811 19918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.685065 19798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:36.685143 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal
uuid: "c675f9ddf30346499d349179d2668d8a"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "3846c1dfbafadcad2a8527b66aa47d3e"
server_key_iv: "376f072985437faaf17d4ce916d037c1"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.685231 19798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:36.704576 19798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:36.706748 19798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:36.711787 19798 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.253:36807
I20260619 18:47:36.711809 19970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.253:36807 every 8 connection(s)
I20260619 18:47:36.712491 19798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
I20260619 18:47:36.714380 19971 sys_catalog.cc:263] Verifying existing consensus state
I20260619 18:47:36.715418 19971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Bootstrap starting.
I20260619 18:47:36.725628 19971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:47:36.725939 19971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Bootstrap complete.
I20260619 18:47:36.726562 19971 raft_consensus.cc:348] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } }
I20260619 18:47:36.726727 19971 raft_consensus.cc:729] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c675f9ddf30346499d349179d2668d8a, State: Initialized, Role: LEARNER
I20260619 18:47:36.726830 19971 consensus_queue.cc:260] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } }
I20260619 18:47:36.727152 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } } }
I20260619 18:47:36.727272 19971 sys_catalog.cc:564] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:36.727362 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: This master's current role is: LEARNER
I20260619 18:47:36.731356 19985 catalog_manager.cc:1259] Loaded cluster ID: dff4f39ee9de43c7843284c70fc51efe
I20260619 18:47:36.731426 19985 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: loading cluster ID for follower catalog manager: success
I20260619 18:47:36.733572 19985 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: acquiring CA information for follower catalog manager: success
I20260619 18:47:36.735051 19985 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 18:47:36.766104 19900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:36.766451 19900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:36.766511 19900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:36.770537 19900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:36.770605 19900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:36.770632 19900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:36.770653 19900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:36.770677 19900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:36.776116 19900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:44503,127.12.234.253:36807,127.12.234.252:37131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.252.keytab
--principal=oryx/127.12.234.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.252:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.234.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:36.777252 19900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:36.778337 19900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:36.786577 19990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.786582 19993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.786621 19991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:36.786787 19900 server_base.cc:1034] running on GCE node
I20260619 18:47:36.787395 19900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:36.788542 19900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:36.789718 19900 hybrid_clock.cc:648] HybridClock initialized: now 1781894856789688 us; error 42 us; skew 500 ppm
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:36.795917 19900 init.cc:377] Logged in from keytab as oryx/127.12.234.252@KRBTEST.COM (short username oryx)
I20260619 18:47:36.796864 19900 webserver.cc:460] Webserver started at http://127.12.234.252:39847/ using document root <none> and password file <none>
I20260619 18:47:36.797399 19900 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:36.797464 19900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:36.797641 19900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:36.799746 19900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data/instance:
uuid: "6e128141cfe0477fbc60a094851bd867"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "6775324f4edd7bb406b625e7dd8b1822"
server_key_iv: "b47322cea7b53631fc6979d720829649"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.800283 19900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal/instance:
uuid: "6e128141cfe0477fbc60a094851bd867"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "6775324f4edd7bb406b625e7dd8b1822"
server_key_iv: "b47322cea7b53631fc6979d720829649"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.804204 19900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:36.806746 20000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.808004 19900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:36.808128 19900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal
uuid: "6e128141cfe0477fbc60a094851bd867"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "6775324f4edd7bb406b625e7dd8b1822"
server_key_iv: "b47322cea7b53631fc6979d720829649"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.808229 19900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:36.833240 19900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:36.836687 19900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:36.836884 19900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:36.844270 19900 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.252:37131
I20260619 18:47:36.844517 20052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.252:37131 every 8 connection(s)
I20260619 18:47:36.845330 19900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
I20260619 18:47:36.848286 20053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:36.852931 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 19900
I20260619 18:47:36.853338 20053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.252@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.252@KRBTEST.COM
I20260619 18:47:36.865628 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.855286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33172 (local address 127.12.234.254:44503)
0619 18:47:36.855413 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.855417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.855653 (+ 236us) server_negotiation.cc:407] Connection header received
0619 18:47:36.856276 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.856279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.856332 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.856400 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.857254 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.857772 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.858840 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.859064 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.859152 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.861777 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.861781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.861784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.861818 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.863457 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.864051 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.864053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.864055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.864174 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.864484 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.864486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.864487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.864696 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.864792 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.865333 (+ 541us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.865459 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 18:47:36.866740 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.855153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:33172)
0619 18:47:36.855532 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.855572 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.856089 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.856461 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.856466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.856846 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.857115 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.857129 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.857888 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.857891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.858730 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.858739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.859202 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.859205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.859372 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.859933 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.859948 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.861608 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.863590 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.863597 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.863608 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.863965 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.864260 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.864262 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.864263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.864396 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.864793 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.864800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.865114 (+ 314us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.865281 (+ 167us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.865655 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":128,"threads_started":1}
I20260619 18:47:36.866740 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.854761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46478 (local address 127.12.234.252:37131)
0619 18:47:36.855217 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.855236 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.855280 (+ 44us) server_negotiation.cc:407] Connection header received
0619 18:47:36.856084 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.856113 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.856307 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.856470 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.857303 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.858144 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.859011 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.859368 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.859459 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.862167 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.862174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.862214 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.862238 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.864011 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.864467 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.864473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.864474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.864544 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.864790 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.864792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.864794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.865148 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.865333 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.865535 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.865724 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":305,"thread_start_us":139,"threads_started":1}
I20260619 18:47:36.868052 20053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.252@KRBTEST.COM for oryx/127.12.234.253@KRBTEST.COM
I20260619 18:47:36.872776 19525 heartbeater.cc:503] Master 127.12.234.254:44503 was elected leader, sending a full tablet report...
I20260619 18:47:36.876065 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.868520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.0.0.1:48658)
0619 18:47:36.868781 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.868795 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.868859 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.869089 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.869092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.869263 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.869353 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.869359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.870258 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.870262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.870813 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.870820 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.871246 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.871249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.871340 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.871754 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.871764 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.872970 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.874721 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.874724 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.874726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.874912 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.875163 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.875165 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.875167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.875277 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.875636 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.875639 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.875738 (+ 99us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.875819 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.875940 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260619 18:47:36.876070 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.868615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48658 (local address 127.12.234.253:36807)
0619 18:47:36.868844 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.868848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.868860 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:47:36.868942 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.868944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.868983 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.869060 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.869462 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.870135 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.870920 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.871151 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.871239 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.873149 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.873153 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.873156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.873185 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.874624 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.874992 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.874994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.874995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.875065 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.875359 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.875362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.875363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.875553 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.875630 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.875853 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.875948 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
I20260619 18:47:36.876972 20053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.252@KRBTEST.COM for oryx/127.12.234.252@KRBTEST.COM
I20260619 18:47:36.884791 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.877579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.12.234.252:37131)
0619 18:47:36.877710 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.877713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.877725 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:47:36.877752 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.877754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.877787 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.877851 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.878234 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.878740 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.879458 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.879686 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.879763 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.881738 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.881742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.881744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.881764 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.883228 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.883645 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.883648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.883650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.883721 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.883971 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.883975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.883977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.884156 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.884246 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.884486 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.884603 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 18:47:36.884810 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.877445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.0.0.1:46490)
0619 18:47:36.877569 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.877583 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.877656 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.877874 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.877875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.878002 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.878074 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.878078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.878838 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.878841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.879355 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.879362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.879773 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.879776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.879829 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.880350 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.880361 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.881604 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.883338 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.883342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.883344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.883548 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.883800 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.883803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.883805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.883883 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.884242 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.884245 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.884334 (+ 89us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.884461 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.884676 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":80}
I20260619 18:47:36.888397 20053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: Bootstrap starting.
I20260619 18:47:36.890861 20053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:36.891642 20053 log.cc:824] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:36.893640 20053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: No bootstrap required, opened a new log
I20260619 18:47:36.896754 20053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } }
I20260619 18:47:36.896991 20053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:36.897034 20053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e128141cfe0477fbc60a094851bd867, State: Initialized, Role: FOLLOWER
I20260619 18:47:36.897436 20053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } }
I20260619 18:47:36.897874 20059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } } }
I20260619 18:47:36.898051 20059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:47:36.898684 20053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 18:47:36.903364 20070 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:47:36.903441 20070 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:47:36.906819 19900 master_runner.cc:428] Detected that this master 6e128141cfe0477fbc60a094851bd867 is joining an existing cluster
I20260619 18:47:36.906895 19900 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.234.252:37131
I20260619 18:47:36.918169 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.909281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:33180)
0619 18:47:36.909698 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.909715 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.909867 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.910351 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.910354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.910627 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.910764 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.910772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.911800 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.911803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.912863 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.912871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.913526 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.913529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.913621 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.914097 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.914112 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.914638 (+ 526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.916393 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.916396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.916399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.916654 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.916934 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.916937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.916939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.917036 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.917546 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.917550 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.917670 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.917780 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.917946 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":39,"spinlock_wait_cycles":3968,"thread_start_us":93,"threads_started":1}
I20260619 18:47:36.918574 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.909370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33180 (local address 127.12.234.254:44503)
0619 18:47:36.909504 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.909509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.909790 (+ 281us) server_negotiation.cc:407] Connection header received
0619 18:47:36.910028 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.910032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.910125 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.910226 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.910898 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.911682 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.912969 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.913300 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.913386 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.914772 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.914776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.914779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.914804 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.916285 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.916752 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.916756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.916758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.916841 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.917165 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.917169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.917171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.917404 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.917501 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.918299 (+ 798us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.918420 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 18:47:36.918983 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.909337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48660 (local address 127.12.234.253:36807)
0619 18:47:36.909623 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.909628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.910130 (+ 502us) server_negotiation.cc:407] Connection header received
0619 18:47:36.910317 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.910320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.910369 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.910416 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.911413 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.912282 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.914145 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.914500 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.914647 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.915787 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.915791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.915794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.915820 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.917366 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.917739 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.917742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.917744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.917819 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.918061 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.918064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.918066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.918295 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.918381 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.918720 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.918847 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260619 18:47:36.919415 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.909434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.0.0.1:48660)
0619 18:47:36.910054 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.910065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.910220 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.910541 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.910543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.911132 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.911267 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.911275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.912473 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.912476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.913973 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.913984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.914637 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.914641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.914742 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.915190 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.915206 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.915651 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.917462 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.917465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.917467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.917654 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.917902 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.917905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.917908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.917985 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.918424 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.918427 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.918533 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.919116 (+ 583us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.919258 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":24,"thread_start_us":94,"threads_started":1}
I20260619 18:47:36.920154 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 19396
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894855, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.254@KRBTEST.COM for oryx/127.12.234.252@KRBTEST.COM
I20260619 18:47:36.928512 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:42443
--local_ip_for_outbound_sockets=127.12.234.193
--tserver_master_addrs=127.12.234.254:44503
--webserver_port=45409
--webserver_interface=127.12.234.193
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab
--principal=oryx/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.234.254:44503,127.12.234.253:36807
--tserver_master_addrs=127.12.234.254:44503,127.12.234.253:36807,127.12.234.252:37131 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:47:36.930619 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.909818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.12.234.252:37131)
0619 18:47:36.909908 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.909912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.910245 (+ 333us) server_negotiation.cc:407] Connection header received
0619 18:47:36.910390 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.910393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.910430 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.910477 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.911119 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.911820 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.913073 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.913383 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.913440 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.927159 (+ 13719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.927162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.927165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.927185 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.928656 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.929395 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.929398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.929400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.929476 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.929724 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.929727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.929729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.929914 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.929980 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.930278 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.930420 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":35}
I20260619 18:47:36.930675 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.909783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.0.0.1:46496)
0619 18:47:36.910147 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.910161 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.910298 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.910609 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.910612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.910870 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.911001 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.911010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.912031 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.912036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.912991 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.912996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.913619 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.913622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.913716 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.926571 (+ 12855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.926586 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.927022 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.929046 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.929049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.929052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.929303 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.929564 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.929567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.929568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.929642 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.930029 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.930031 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.930110 (+ 79us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.930390 (+ 280us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.930489 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":40,"thread_start_us":80,"threads_started":1}
I20260619 18:47:36.932578 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.922684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.0.0.1:46510)
0619 18:47:36.922956 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.922970 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.923050 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.923542 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.923545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.923728 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.923832 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.923839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.924863 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.924866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.925651 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.925661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.926290 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.926294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.926404 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.926818 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.926834 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.928533 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.931063 (+ 2530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.931067 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.931069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.931332 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.931648 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.931651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.931652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.931752 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.932106 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.932109 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.932199 (+ 90us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.932281 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.932422 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260619 18:47:36.932605 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.922922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46510 (local address 127.12.234.252:37131)
0619 18:47:36.923264 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.923274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.923287 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:47:36.923354 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.923357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.923410 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.923503 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.923954 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.924760 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.925757 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.926118 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.926629 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.928726 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.928731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.928733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.928758 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.930935 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.931460 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.931463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.931465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.931548 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.931839 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.931842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.931844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.932011 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.932090 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.932341 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.932437 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":121,"threads_started":1}
I20260619 18:47:36.933540 19334 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.234.252:37131: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6e128141cfe0477fbc60a094851bd867"
member_type: NON_VOTER
last_known_addr {
host: "127.12.234.252"
port: 37131
}
attrs {
promote: true
}
}
dest_uuid: "bfa9f9d9741744ca8c9985b264c1d108"
cas_config_opid_index: 5
I20260619 18:47:36.934250 19334 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } }
I20260619 18:47:36.935621 19894 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6e128141cfe0477fbc60a094851bd867 (127.12.234.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.936239 20027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:36.937053 19894 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.234.252:37131
I20260619 18:47:36.937229 19894 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.937331 19894 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:36.937801 19900 master.cc:537] Master@127.12.234.252:37131 shutting down...
I20260619 18:47:36.938170 20027 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Refusing update from remote peer bfa9f9d9741744ca8c9985b264c1d108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 18:47:36.939554 19895 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 18:47:36.941447 19318 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 -> Peer 6e128141cfe0477fbc60a094851bd867 (127.12.234.252:37131): Couldn't send request to peer 6e128141cfe0477fbc60a094851bd867. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 18:47:36.943115 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.936144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.12.234.253:36807)
0619 18:47:36.936261 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.936266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.936523 (+ 257us) server_negotiation.cc:407] Connection header received
0619 18:47:36.936577 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.936580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.936641 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.936725 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:36.937120 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.937860 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.939837 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.940494 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.940598 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.940758 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.942909 (+ 2151us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.943006 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:47:36.943403 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.935833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.0.0.1:48670)
0619 18:47:36.936350 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.936364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.936445 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.936865 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.936868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.936884 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:36.936981 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.936988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.937998 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.938001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.939241 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.939254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.942132 (+ 2878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.942137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.942332 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.942401 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.942475 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.942563 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448}
I20260619 18:47:36.943459 19900 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:36.943745 19900 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:36.943830 19900 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: stopping tablet replica
I20260619 18:47:36.944728 19945 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 LEARNER]: Refusing update from remote peer bfa9f9d9741744ca8c9985b264c1d108: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 18:47:36.945200 19895 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 18:47:36.947352 19945 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6e128141cfe0477fbc60a094851bd867 (127.12.234.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.949052 20082 sys_catalog.cc:455] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa9f9d9741744ca8c9985b264c1d108. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.949178 20082 sys_catalog.cc:458] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: This master's current role is: LEARNER
I20260619 18:47:36.949148 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: NON_VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: true } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.949220 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: This master's current role is: LEARNER
I20260619 18:47:36.950690 19895 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: attempting to promote NON_VOTER c675f9ddf30346499d349179d2668d8a to VOTER
I20260619 18:47:36.951122 19895 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } }
I20260619 18:47:36.952348 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.952065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.0.0.1:46514)
0619 18:47:36.952185 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.952268 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 18:47:36.952512 19945 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 LEARNER]: Refusing update from remote peer bfa9f9d9741744ca8c9985b264c1d108: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 18:47:36.953141 19893 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 18:47:36.953241 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.953031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.0.0.1:46522)
0619 18:47:36.953134 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.953184 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 18:47:36.953338 19318 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 -> Peer 6e128141cfe0477fbc60a094851bd867 (127.12.234.252:37131): Couldn't send request to peer 6e128141cfe0477fbc60a094851bd867. Status: Network error: Client connection negotiation failed: client connection to 127.12.234.252:37131: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 18:47:36.954713 19894 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c675f9ddf30346499d349179d2668d8a (127.12.234.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.955888 19894 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.955994 19894 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:36.956118 19894 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.956171 19894 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:36.956674 19945 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c675f9ddf30346499d349179d2668d8a (127.12.234.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.957938 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:36.958042 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:47:36.960635 19900 master.cc:552] Master@127.12.234.252:37131 shutdown complete.
I20260619 18:47:36.960680 19900 master_runner.cc:305] Clearing existing system tablet
I20260619 18:47:36.960889 19900 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:36.960935 19900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:36.961946 19900 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:36.962805 20084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.963024 19900 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:36.963151 19900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 18:47:36.963191 19900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal
uuid: "6e128141cfe0477fbc60a094851bd867"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "6775324f4edd7bb406b625e7dd8b1822"
server_key_iv: "b47322cea7b53631fc6979d720829649"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.964845 19900 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:36.966823 19900 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 18:47:36.966909 19900 log.cc:1196] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 18:47:36.967156 19900 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: Deleting consensus metadata
I20260619 18:47:36.967310 19900 master_runner.cc:315] Copying system tablet from 127.12.234.254:44503
I20260619 18:47:36.968351 19900 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: tablet copy: Beginning tablet copy session from remote peer at address 127.12.234.254:44503
I20260619 18:47:36.975641 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.968968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.12.234.254:44503)
0619 18:47:36.969094 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:36.969099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:36.969279 (+ 180us) server_negotiation.cc:407] Connection header received
0619 18:47:36.969395 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:36.969398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:36.969447 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:36.969513 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:36.969946 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.970536 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.971336 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.971555 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.971650 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.972714 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:36.972718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:36.972721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:36.972747 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:36.974070 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.974500 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.974503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.974505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.974585 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:36.974832 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:36.974836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:36.974837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:36.975019 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:36.975102 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:36.975366 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 18:47:36.975472 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:47:36.975662 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:36.968877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:33194)
0619 18:47:36.969191 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:36.969202 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:36.969314 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:36.969537 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:36.969539 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:36.969732 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:36.969829 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.969834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.970636 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.970640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.971227 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:36.971235 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:36.971657 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:36.971660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:36.971736 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:36.972234 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:36.972248 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:36.972571 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:36.974171 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.974174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.974216 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.974411 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.974671 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:36.974674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:36.974677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:36.974757 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:36.975109 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:36.975112 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:36.975217 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 18:47:36.975303 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 18:47:36.975450 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":91,"threads_started":1}
I20260619 18:47:36.976356 19354 tablet_copy_service.cc:140] P bfa9f9d9741744ca8c9985b264c1d108: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6e128141cfe0477fbc60a094851bd867 ({username='oryx', principal='oryx/127.12.234.252@KRBTEST.COM'} at 127.0.0.1:33194)
I20260619 18:47:36.976446 19354 tablet_copy_service.cc:161] P bfa9f9d9741744ca8c9985b264c1d108: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6e128141cfe0477fbc60a094851bd867 at {username='oryx', principal='oryx/127.12.234.252@KRBTEST.COM'} at 127.0.0.1:33194: session id = 6e128141cfe0477fbc60a094851bd867-00000000000000000000000000000000
I20260619 18:47:36.977260 19354 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 18:47:36.978477 19900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 18:47:36.980315 19900 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: tablet copy: Starting download of 0 data blocks...
I20260619 18:47:36.980509 19900 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: tablet copy: Starting download of 1 WAL segments...
I20260619 18:47:36.981858 19900 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 18:47:36.983633 19354 tablet_copy_service.cc:342] P bfa9f9d9741744ca8c9985b264c1d108: Request end of tablet copy session 6e128141cfe0477fbc60a094851bd867-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.234.252@KRBTEST.COM'} at 127.0.0.1:33194
I20260619 18:47:36.983714 19354 tablet_copy_service.cc:433] P bfa9f9d9741744ca8c9985b264c1d108: ending tablet copy session 6e128141cfe0477fbc60a094851bd867-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6e128141cfe0477fbc60a094851bd867
I20260619 18:47:36.990782 19900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:36.992501 20091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:36.992699 19900 server_base.cc:1034] running on GCE node
W20260619 18:47:36.992751 20094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:36.992828 20092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:36.993170 19900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:36.993472 19900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:36.994606 19900 hybrid_clock.cc:648] HybridClock initialized: now 1781894856994596 us; error 28 us; skew 500 ppm
Jun 19 18:47:36 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894856, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:36.995968 19900 init.cc:377] Logged in from keytab as oryx/127.12.234.252@KRBTEST.COM (short username oryx)
I20260619 18:47:36.996431 19900 webserver.cc:460] Webserver started at http://127.12.234.252:40895/ using document root <none> and password file <none>
I20260619 18:47:36.996624 19900 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:36.996658 19900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:36.997732 19900 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:36.998620 20101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:36.998826 19900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:36.998896 19900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal
uuid: "6e128141cfe0477fbc60a094851bd867"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "6775324f4edd7bb406b625e7dd8b1822"
server_key_iv: "b47322cea7b53631fc6979d720829649"
server_key_version: "encryptionkey@0"
I20260619 18:47:36.998982 19900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:37.003691 19900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:37.005538 19900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:37.010088 19900 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.252:37131
I20260619 18:47:37.010731 19900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
I20260619 18:47:37.012179 20154 sys_catalog.cc:263] Verifying existing consensus state
I20260619 18:47:37.012996 20154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: Bootstrap starting.
I20260619 18:47:37.021929 20154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:47:37.022285 20154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: Bootstrap complete.
I20260619 18:47:37.022361 20153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.252:37131 every 8 connection(s)
I20260619 18:47:37.022902 20154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } }
I20260619 18:47:37.023028 20154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6e128141cfe0477fbc60a094851bd867, State: Initialized, Role: LEARNER
I20260619 18:47:37.023135 20154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } }
I20260619 18:47:37.023455 20157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:37.023699 20157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: This master's current role is: LEARNER
I20260619 18:47:37.023618 20154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:37.026430 20168 catalog_manager.cc:1259] Loaded cluster ID: dff4f39ee9de43c7843284c70fc51efe
I20260619 18:47:37.026554 20168 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: loading cluster ID for follower catalog manager: success
I20260619 18:47:37.028740 20168 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: acquiring CA information for follower catalog manager: success
I20260619 18:47:37.029935 20168 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 18:47:37.094799 19659 heartbeater.cc:503] Master 127.12.234.254:44503 was elected leader, sending a full tablet report...
W20260619 18:47:37.103145 20080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:37.103545 20080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:37.103615 20080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:37.109858 20080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:37.109975 20080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:37.110081 20080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:37.118106 20080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab
--principal=oryx/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:42443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=45409
--tserver_master_addrs=127.12.234.254:44503,127.12.234.253:36807,127.12.234.252:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:37.119608 20080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:37.121078 20080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:37.132512 20174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:37.132469 20176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:37.132906 20173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:37.132889 20080 server_base.cc:1034] running on GCE node
I20260619 18:47:37.133474 20080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:37.134243 20080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:37.135432 20080 hybrid_clock.cc:648] HybridClock initialized: now 1781894857135407 us; error 36 us; skew 500 ppm
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:37.138423 20080 init.cc:377] Logged in from keytab as oryx/127.12.234.193@KRBTEST.COM (short username oryx)
I20260619 18:47:37.139464 20080 webserver.cc:460] Webserver started at http://127.12.234.193:45409/ using document root <none> and password file <none>
I20260619 18:47:37.140067 20080 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:37.140147 20080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:37.144317 20080 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:47:37.146463 20183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:37.147786 20080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 18:47:37.147890 20080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "e175e83617ab4b298c7abd17b946b6d2"
format_stamp: "Formatted at 2026-06-19 18:47:35 on dist-test-slave-fljv"
server_key: "c6d1d235bd1782e1d112dbcf400689c6"
server_key_iv: "2cf64028a8310876a4c8f4de23a34519"
server_key_version: "encryptionkey@0"
I20260619 18:47:37.148450 20080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:37.159878 20080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:37.163160 20080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:37.163335 20080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:37.163944 20080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:37.164803 20080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:37.164849 20080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:37.164888 20080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:37.164914 20080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:37.173574 20080 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:42443
I20260619 18:47:37.173595 20296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:42443 every 8 connection(s)
I20260619 18:47:37.174604 20080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:37.176092 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 20080
I20260619 18:47:37.176263 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 19530
I20260619 18:47:37.184370 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:38539
--local_ip_for_outbound_sockets=127.12.234.194
--tserver_master_addrs=127.12.234.254:44503
--webserver_port=35369
--webserver_interface=127.12.234.194
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab
--principal=oryx/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.234.254:44503,127.12.234.253:36807
--tserver_master_addrs=127.12.234.254:44503,127.12.234.253:36807,127.12.234.252:37131 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.252@KRBTEST.COM
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.253@KRBTEST.COM
I20260619 18:47:37.192155 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.178987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:57541 (local address 127.12.234.253:36807)
0619 18:47:37.179110 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.179114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.179485 (+ 371us) server_negotiation.cc:407] Connection header received
0619 18:47:37.179643 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.179646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.179695 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.179741 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.181395 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.182248 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.184012 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.184318 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.184396 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.188578 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.188582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.188584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.188616 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.190304 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.190748 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.190752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.190754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.190843 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.191144 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.191147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.191149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.191344 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.191411 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.191830 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.191987 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 18:47:37.192212 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.178500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:51427 (local address 127.12.234.252:37131)
0619 18:47:37.179215 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.179228 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.179242 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:37.179284 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.179287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.179333 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.179836 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.182324 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.183394 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.184481 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.184800 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.185351 (+ 551us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.187831 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.187835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.187839 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.187870 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.189597 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.190291 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.190294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.190296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.190387 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.190783 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.190786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.190787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.191024 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.191092 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.191830 (+ 738us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.191983 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":71,"threads_started":1}
I20260619 18:47:37.192543 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.176385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:48233 (local address 127.12.234.254:44503)
0619 18:47:37.176908 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.176912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.178065 (+ 1153us) server_negotiation.cc:407] Connection header received
0619 18:47:37.179241 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.179244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.179308 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.179362 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.181652 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.182432 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.184017 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.184353 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.184422 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.188105 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.188109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.188112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.188405 (+ 293us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.190628 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.191158 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.191162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.191164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.191259 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.191565 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.191569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.191571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.191792 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.191862 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.192254 (+ 392us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.192383 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435}
I20260619 18:47:37.194048 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.178409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.12.234.193:51427)
0619 18:47:37.178732 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.178746 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.178984 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.179664 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.179671 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.180337 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:37.181231 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.181253 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.183534 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.183537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.184320 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.184329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.184923 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.184927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.185023 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.185544 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:37.185557 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:37.187591 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:37.189742 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.189750 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.189771 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.190131 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.190502 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.190505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.190507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.190688 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.191166 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:37.191173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:37.191540 (+ 367us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.192895 (+ 1355us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.193016 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":117,"spinlock_wait_cycles":15744,"thread_start_us":121,"threads_started":1}
I20260619 18:47:37.194340 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.178864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.12.234.193:57541)
0619 18:47:37.179409 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.179420 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.179546 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.179871 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.179874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.180337 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:37.181231 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.181253 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.182409 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.182412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.183877 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.183888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.184987 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.184991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.185572 (+ 581us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.186054 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:37.186067 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:37.188406 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:37.190412 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.190415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.190417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.190635 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.190951 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.190954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.190956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.191047 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.191478 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:37.191480 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:37.191551 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.192404 (+ 853us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.192837 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":142,"spinlock_wait_cycles":8320,"thread_start_us":86,"threads_started":1}
I20260619 18:47:37.194049 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.176849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:48233)
0619 18:47:37.177877 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.177935 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.178984 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.179662 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.179671 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.181438 (+ 1767us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:37.181539 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.181543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.182560 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.182563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.183874 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.183884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.184610 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.184613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.184801 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.185446 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:37.185467 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:37.187921 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:37.190775 (+ 2854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.190778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.190780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.191033 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.191376 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.191379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.191381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.191465 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.191945 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:37.191948 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:37.192059 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.192760 (+ 701us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.192914 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":68,"thread_start_us":634,"threads_started":1}
I20260619 18:47:37.195803 20299 heartbeater.cc:347] Connected to a master server at 127.12.234.253:36807
I20260619 18:47:37.195850 20297 heartbeater.cc:347] Connected to a master server at 127.12.234.252:37131
I20260619 18:47:37.195937 20298 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44503
I20260619 18:47:37.196091 20297 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:37.196091 20298 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:37.196091 20299 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:37.196648 20298 heartbeater.cc:511] Master 127.12.234.254:44503 requested a full tablet report, sending...
I20260619 18:47:37.196652 20299 heartbeater.cc:511] Master 127.12.234.253:36807 requested a full tablet report, sending...
I20260619 18:47:37.196651 20297 heartbeater.cc:511] Master 127.12.234.252:37131 requested a full tablet report, sending...
I20260619 18:47:37.197929 19334 ts_manager.cc:194] Re-registered known tserver with Master: e175e83617ab4b298c7abd17b946b6d2 (127.12.234.193:42443)
I20260619 18:47:37.198341 20118 ts_manager.cc:194] Registered new tserver with Master: e175e83617ab4b298c7abd17b946b6d2 (127.12.234.193:42443)
I20260619 18:47:37.198741 19334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:48233
I20260619 18:47:37.198925 19935 ts_manager.cc:194] Registered new tserver with Master: e175e83617ab4b298c7abd17b946b6d2 (127.12.234.193:42443)
W20260619 18:47:37.302495 20305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:37.302781 20305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:37.302824 20305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:37.307242 20305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:37.307325 20305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:37.307390 20305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:37.313256 20305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab
--principal=oryx/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:38539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=35369
--tserver_master_addrs=127.12.234.254:44503,127.12.234.253:36807,127.12.234.252:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:37.313766 19793 heartbeater.cc:503] Master 127.12.234.254:44503 was elected leader, sending a full tablet report...
I20260619 18:47:37.314503 20305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:37.315711 20305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:37.325577 20310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:37.325753 20305 server_base.cc:1034] running on GCE node
W20260619 18:47:37.325584 20311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:37.325659 20313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:37.326315 20305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:37.327090 20305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:37.328279 20305 hybrid_clock.cc:648] HybridClock initialized: now 1781894857328247 us; error 45 us; skew 500 ppm
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:37.331285 20305 init.cc:377] Logged in from keytab as oryx/127.12.234.194@KRBTEST.COM (short username oryx)
I20260619 18:47:37.332271 20305 webserver.cc:460] Webserver started at http://127.12.234.194:35369/ using document root <none> and password file <none>
I20260619 18:47:37.332856 20305 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:37.332939 20305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:37.337186 20305 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:37.339485 20320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:37.341015 20305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:37.341154 20305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "663ae2a2502b7f94cbb0e37f96a4979e"
server_key_iv: "1736821f5a3ae626757cf10746495b8b"
server_key_version: "encryptionkey@0"
I20260619 18:47:37.341740 20305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:37.366730 20305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:37.370105 20305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:37.370323 20305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:37.370936 20305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:37.371832 20305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:37.371878 20305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:37.371929 20305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:37.371958 20305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:37.380609 20305 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:38539
I20260619 18:47:37.380685 20433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:38539 every 8 connection(s)
I20260619 18:47:37.381672 20305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.253@KRBTEST.COM
I20260619 18:47:37.391891 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 20305
I20260619 18:47:37.392086 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 19664
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.252@KRBTEST.COM
I20260619 18:47:37.397507 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.383493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:45831 (local address 127.12.234.254:44503)
0619 18:47:37.383624 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.383628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.384592 (+ 964us) server_negotiation.cc:407] Connection header received
0619 18:47:37.385698 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.385702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.385761 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.385834 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.387248 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.388060 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.390323 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.390595 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.390699 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.393474 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.393479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.393482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.393513 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.395176 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.395661 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.395665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.395667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.395749 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.396117 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.396121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.396122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.396358 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.396427 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.397187 (+ 760us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.397320 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 18:47:37.397866 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.385249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:57303 (local address 127.12.234.253:36807)
0619 18:47:37.385375 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.385379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.385391 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:47:37.385962 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.385965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.386014 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.386087 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.387265 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.387781 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.389414 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.389773 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.389856 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.392697 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.392702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.392705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.392735 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.394491 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.396360 (+ 1869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.396364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.396366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.396481 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.396819 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.396823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.396825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.397081 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.397167 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.397560 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.397694 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:47:37.398288 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.383493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:44433 (local address 127.12.234.252:37131)
0619 18:47:37.383629 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.383634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.384588 (+ 954us) server_negotiation.cc:407] Connection header received
0619 18:47:37.385785 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.385787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.385836 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.385900 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.387275 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.388063 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.389483 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.389730 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.389826 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.394137 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.394142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.394145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.394204 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.395896 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.396428 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.396432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.396434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.396526 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.396835 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.396839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.396841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.397059 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.397126 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.397445 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.398101 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 18:47:37.399351 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.383979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.194:45831)
0619 18:47:37.384437 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.384489 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.385457 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.386016 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.386025 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.386593 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:37.387110 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.387123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.389296 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.389299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.390164 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.390172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.390717 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.390721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.390814 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.391166 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:37.391180 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:37.393300 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:37.395276 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.395279 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.395281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.395563 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.395830 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.395833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.395835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.396008 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.396518 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:37.396525 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:37.396896 (+ 371us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.397711 (+ 815us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.398108 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":85,"spinlock_wait_cycles":40576,"thread_start_us":115,"threads_started":1}
I20260619 18:47:37.399365 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.383979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.12.234.194:44433)
0619 18:47:37.384437 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.384489 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.385463 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.386017 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.386024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.386621 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:37.387112 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.387126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.388169 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.388172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.389354 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.389365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.389881 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.389886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.390071 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.391456 (+ 1385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:37.391469 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:37.393917 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:37.396019 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.396022 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.396024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.396313 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.396627 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.396630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.396631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.396739 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.397194 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:37.397197 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:37.397275 (+ 78us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.398583 (+ 1308us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.398724 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":144,"spinlock_wait_cycles":23424,"thread_start_us":104,"threads_started":1}
I20260619 18:47:37.399654 20440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.384949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.12.234.194:57303)
0619 18:47:37.385207 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.385221 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.385458 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.386114 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.386117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.386621 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:37.387112 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.387126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.387925 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.387927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.389289 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.389298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.390006 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.390010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.390152 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.390770 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:37.390794 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:37.392447 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:37.394623 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.394631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.395019 (+ 388us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.396228 (+ 1209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.396606 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.396609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.396611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.396712 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.397249 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:37.397252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:37.397349 (+ 97us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.398083 (+ 734us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.398263 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":114,"spinlock_wait_cycles":2944,"thread_start_us":76,"threads_started":1}
I20260619 18:47:37.400863 20435 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44503
I20260619 18:47:37.400960 20434 heartbeater.cc:347] Connected to a master server at 127.12.234.252:37131
I20260619 18:47:37.400880 20436 heartbeater.cc:347] Connected to a master server at 127.12.234.253:36807
I20260619 18:47:37.401129 20436 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:37.401129 20434 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:37.401129 20435 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:37.401713 20434 heartbeater.cc:511] Master 127.12.234.252:37131 requested a full tablet report, sending...
I20260619 18:47:37.401736 20436 heartbeater.cc:511] Master 127.12.234.253:36807 requested a full tablet report, sending...
I20260619 18:47:37.401773 20435 heartbeater.cc:511] Master 127.12.234.254:44503 requested a full tablet report, sending...
I20260619 18:47:37.402349 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:42317
--local_ip_for_outbound_sockets=127.12.234.195
--tserver_master_addrs=127.12.234.254:44503
--webserver_port=34305
--webserver_interface=127.12.234.195
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab
--principal=oryx/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.234.254:44503,127.12.234.253:36807
--tserver_master_addrs=127.12.234.254:44503,127.12.234.253:36807,127.12.234.252:37131 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:47:37.402886 19334 ts_manager.cc:194] Re-registered known tserver with Master: 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539)
I20260619 18:47:37.403088 19935 ts_manager.cc:194] Registered new tserver with Master: 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539)
I20260619 18:47:37.403512 19334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:45831
I20260619 18:47:37.404245 20118 ts_manager.cc:194] Registered new tserver with Master: 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539)
I20260619 18:47:37.520462 20445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.516212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.0.0.1:46536)
0619 18:47:37.516524 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.516540 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.516665 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.517043 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.517045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.517060 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:37.517159 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.517164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.518165 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.518168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.519051 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.519065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.520054 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.520057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.520159 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.520184 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.520258 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.520332 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":73,"threads_started":1}
I20260619 18:47:37.520485 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.516325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46536 (local address 127.12.234.252:37131)
0619 18:47:37.516471 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.516477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.516620 (+ 143us) server_negotiation.cc:407] Connection header received
0619 18:47:37.516771 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.516775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.516855 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.516990 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:37.517258 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.518027 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.519177 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.519858 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.519984 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.520147 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.520284 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.520363 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 18:47:37.520447 20441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:37.520841 20441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:37.520913 20441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 18:47:37.521302 20124 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 18:47:37.521739 20157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa9f9d9741744ca8c9985b264c1d108. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:37.521875 20157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: This master's current role is: LEARNER
W20260619 18:47:37.526158 20441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:37.526269 20441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:37.526347 20441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:37.532100 20441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab
--principal=oryx/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:42317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=34305
--tserver_master_addrs=127.12.234.254:44503,127.12.234.253:36807,127.12.234.252:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:37.533221 20441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:37.534298 20441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:37.544097 20450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:37.544209 20441 server_base.cc:1034] running on GCE node
W20260619 18:47:37.544041 20448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:37.544041 20447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:37.544792 20441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:37.545503 20441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:37.546680 20441 hybrid_clock.cc:648] HybridClock initialized: now 1781894857546638 us; error 49 us; skew 500 ppm
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:37.549561 20441 init.cc:377] Logged in from keytab as oryx/127.12.234.195@KRBTEST.COM (short username oryx)
I20260619 18:47:37.550591 20441 webserver.cc:460] Webserver started at http://127.12.234.195:34305/ using document root <none> and password file <none>
I20260619 18:47:37.551158 20441 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:37.551234 20441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:37.555632 20441 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:37.557894 20457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:37.559451 20441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:37.559576 20441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "3ff50c2d32a5412d8af73c53de93d3cf"
format_stamp: "Formatted at 2026-06-19 18:47:36 on dist-test-slave-fljv"
server_key: "25b835f3c01d74f33f7ded7035453c1a"
server_key_iv: "bcc40b09b4bfe6ed65d6bd198c745067"
server_key_version: "encryptionkey@0"
I20260619 18:47:37.560171 20441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:37.577797 20441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:37.581702 20441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:37.581897 20441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:37.582590 20441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:37.583580 20441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:37.583629 20441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:37.583671 20441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:37.583698 20441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:37.592629 20441 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:42317
I20260619 18:47:37.592664 20570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:42317 every 8 connection(s)
I20260619 18:47:37.593709 20441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:37.600641 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 20441
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.253@KRBTEST.COM
Jun 19 18:47:37 dist-test-slave-fljv krb5kdc[19289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894857, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.252@KRBTEST.COM
I20260619 18:47:37.612682 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.597355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.12.234.195:37167)
0619 18:47:37.597607 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.597619 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.597900 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.598471 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.598481 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.599036 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:37.599727 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.599745 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.601814 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.601817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.602747 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.602767 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.603431 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.603435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.603520 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.603920 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:37.603934 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:37.606866 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:37.609114 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.609121 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.609142 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.609524 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.609839 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.609842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.609843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.610057 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.610551 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:37.610558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:37.610939 (+ 381us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.611169 (+ 230us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.611518 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":89,"spinlock_wait_cycles":3840,"thread_start_us":90,"threads_started":1}
I20260619 18:47:37.613381 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.597471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:37167 (local address 127.12.234.253:36807)
0619 18:47:37.597602 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.597605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.597689 (+ 84us) server_negotiation.cc:407] Connection header received
0619 18:47:37.598109 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.598112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.598167 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.598270 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.600931 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.601710 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.602894 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.603203 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.603281 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.607340 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.607344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.607347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.607376 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.608981 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.609642 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.609646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.609647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.609738 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.610163 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.610167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.610169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.610412 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.610496 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.613110 (+ 2614us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.613234 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 18:47:37.614266 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.595652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:60605 (local address 127.12.234.252:37131)
0619 18:47:37.595776 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.595780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.596968 (+ 1188us) server_negotiation.cc:407] Connection header received
0619 18:47:37.598143 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.598147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.598235 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.598289 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.599907 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.600775 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.603210 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.603975 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.604055 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.610586 (+ 6531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.610590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.610592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.610618 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.612294 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.612737 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.612740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.612742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.612830 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.613396 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.613399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.613401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.613608 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.613663 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.613977 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.614097 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:47:37.614643 20573 heartbeater.cc:347] Connected to a master server at 127.12.234.253:36807
I20260619 18:47:37.614835 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.596244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.12.234.195:60605)
0619 18:47:37.596788 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.596852 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.597901 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.598471 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.598481 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.599041 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:37.599734 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.599746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.602289 (+ 2543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.602292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.603083 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.603090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.604206 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.604209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.604298 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.604779 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:37.604792 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:37.610403 (+ 5611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:37.612405 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.612408 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.612410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.612631 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.613212 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.613215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.613217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.613299 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.613731 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:37.613733 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:37.613814 (+ 81us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.614447 (+ 633us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.614598 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":121,"spinlock_wait_cycles":41472,"thread_start_us":156,"threads_started":1}
I20260619 18:47:37.614934 20573 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:37.614977 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.603402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.12.234.252:37131)
0619 18:47:37.604106 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.604110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.604927 (+ 817us) server_negotiation.cc:407] Connection header received
0619 18:47:37.604972 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.604975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.605027 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.605106 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.606393 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.607147 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.608241 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.608602 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.608700 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.610256 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.610260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.610262 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.610290 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.612444 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.613175 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.613178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.613180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.613257 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.613793 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.613796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.613798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.613992 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.614648 (+ 656us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.614722 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.614827 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":73,"threads_started":1}
I20260619 18:47:37.615525 20573 heartbeater.cc:511] Master 127.12.234.253:36807 requested a full tablet report, sending...
I20260619 18:47:37.615773 20571 heartbeater.cc:347] Connected to a master server at 127.12.234.252:37131
I20260619 18:47:37.615854 20571 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:37.616019 20571 heartbeater.cc:511] Master 127.12.234.252:37131 requested a full tablet report, sending...
I20260619 18:47:37.616649 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.595652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:42773 (local address 127.12.234.254:44503)
0619 18:47:37.595776 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.595780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.596969 (+ 1189us) server_negotiation.cc:407] Connection header received
0619 18:47:37.598172 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.598196 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.598250 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.598297 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.599915 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.600775 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.602374 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.602712 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.602791 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.607861 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.607866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.607868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.607895 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.609847 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.612406 (+ 2559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.612409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.612411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.612504 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.612798 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.612802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.612804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.613002 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.613078 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.616368 (+ 3290us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.616487 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:47:37.616882 20110 ts_manager.cc:194] Registered new tserver with Master: 3ff50c2d32a5412d8af73c53de93d3cf (127.12.234.195:42317)
I20260619 18:47:37.617070 19935 ts_manager.cc:194] Registered new tserver with Master: 3ff50c2d32a5412d8af73c53de93d3cf (127.12.234.195:42317)
I20260619 18:47:37.617671 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.596244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.195:42773)
0619 18:47:37.596788 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.596852 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.597898 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.598471 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.598481 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.599041 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:37.599736 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.599746 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.600984 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.600988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.602241 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.602253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.603004 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.603008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.603212 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.603820 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:37.603842 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:37.607704 (+ 3862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:37.610228 (+ 2524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.610231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.610233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.610545 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.612609 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:37.612612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:37.612614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:37.612696 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:37.613132 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:37.613135 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:37.613226 (+ 91us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.617339 (+ 4113us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.617487 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":99,"spinlock_wait_cycles":40576,"thread_start_us":126,"threads_started":1}
I20260619 18:47:37.618207 20572 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44503
I20260619 18:47:37.618291 20572 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:37.618443 20572 heartbeater.cc:511] Master 127.12.234.254:44503 requested a full tablet report, sending...
I20260619 18:47:37.619020 19334 ts_manager.cc:194] Re-registered known tserver with Master: 3ff50c2d32a5412d8af73c53de93d3cf (127.12.234.195:42317)
I20260619 18:47:37.619609 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.603194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48678 (local address 127.12.234.253:36807)
0619 18:47:37.603971 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.603975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.605204 (+ 1229us) server_negotiation.cc:407] Connection header received
0619 18:47:37.605244 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.605247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.605292 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.605373 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.606455 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.607233 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.609130 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.609500 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.609577 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.615330 (+ 5753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.615334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.615336 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.615362 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.617510 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.618296 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.618300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.618302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.618377 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.618685 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.618688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.618690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.618904 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.619024 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.619326 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.619435 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":80,"threads_started":1}
I20260619 18:47:37.619729 19334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:42773
I20260619 18:47:37.622080 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.604779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.12.234.254:44503)
0619 18:47:37.605307 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.605320 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.606026 (+ 706us) server_negotiation.cc:407] Connection header received
0619 18:47:37.606074 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.606077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.606128 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.606273 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.607114 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.609965 (+ 2851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.611819 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.612174 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.612276 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.619106 (+ 6830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.619109 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.619112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.619133 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.620635 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.621004 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.621007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.621009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.621085 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.621315 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.621318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.621320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.621482 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.621595 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.621802 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.621917 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":58,"threads_started":1}
I20260619 18:47:37.624742 19334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:37.627943 19334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:37.630522 19895 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: NON_VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: true } } }
I20260619 18:47:37.630654 19895 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:37.634524 19895 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: attempting to promote NON_VOTER 6e128141cfe0477fbc60a094851bd867 to VOTER
I20260619 18:47:37.634845 19895 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } }
I20260619 18:47:37.636439 20124 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 LEARNER]: Refusing update from remote peer bfa9f9d9741744ca8c9985b264c1d108: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 18:47:37.636778 19945 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Refusing update from remote peer bfa9f9d9741744ca8c9985b264c1d108: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 18:47:37.636848 20590 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 18:47:37.637315 20590 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 18:47:37.638550 20591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6e128141cfe0477fbc60a094851bd867 (127.12.234.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:37.639153 20124 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6e128141cfe0477fbc60a094851bd867 (127.12.234.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:37.639798 19945 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6e128141cfe0477fbc60a094851bd867 (127.12.234.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:37.640064 20591 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:37.640159 20591 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:37.640275 20591 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:37.640344 20591 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:37.640409 20590 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:37.640513 20590 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:37.640884 20157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:37.640985 20157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:47:37.641968 20599 sys_catalog.cc:455] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa9f9d9741744ca8c9985b264c1d108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:37.642104 20599 sys_catalog.cc:458] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:47:37.651831 20445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.646042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:42443 (local address 127.0.0.1:46656)
0619 18:47:37.646204 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.646219 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.646301 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.648491 (+ 2190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.648494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.648510 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:37.648601 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.648607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.649711 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.649715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.650722 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.650731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.651551 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.651555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.651631 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.651648 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.651690 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.651752 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 18:47:37.651816 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.647044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:42317 (local address 127.0.0.1:55020)
0619 18:47:37.647352 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.647365 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.647441 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.647886 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.647888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.647905 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:37.648016 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.648022 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.649010 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.649013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.650257 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.650267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.651384 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.651388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.651473 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.651510 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.651566 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.651640 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":77,"threads_started":1}
I20260619 18:47:37.651887 20604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.647205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55020 (local address 127.12.234.195:42317)
0619 18:47:37.647474 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.647486 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.647508 (+ 22us) server_negotiation.cc:407] Connection header received
0619 18:47:37.647575 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.647579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.647760 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.647884 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:37.648101 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.648900 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.650383 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.651254 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.651432 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.651597 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.651648 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.651741 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
I20260619 18:47:37.652355 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.646208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.12.234.193:42443)
0619 18:47:37.648166 (+ 1958us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.648178 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.648195 (+ 17us) server_negotiation.cc:407] Connection header received
0619 18:47:37.648250 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.648254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.648374 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.648509 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:37.648723 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.649605 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.650836 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.651403 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.651934 (+ 531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.652071 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.652124 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.652225 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1860,"thread_start_us":80,"threads_started":1}
I20260619 18:47:37.652828 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.646325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:38539 (local address 127.0.0.1:47380)
0619 18:47:37.646615 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.646628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.646861 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.648686 (+ 1825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.648689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.648705 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:37.648765 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.648769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.650274 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.650277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.651386 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.651396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.652459 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.652463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.652554 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.652578 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.652639 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.652708 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":130,"threads_started":1}
I20260619 18:47:37.652897 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.646423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.12.234.194:38539)
0619 18:47:37.647634 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.647643 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.647661 (+ 18us) server_negotiation.cc:407] Connection header received
0619 18:47:37.647721 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.647724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.647888 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.648878 (+ 990us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:37.648988 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.650129 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.651522 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.652326 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.652501 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.652628 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.652676 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.652766 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1126,"thread_start_us":93,"threads_started":1}
I20260619 18:47:37.655433 20231 tablet_service.cc:1467] Processing CreateTablet for tablet 77705a72c4f94451bb000836277117cc (DEFAULT_TABLE table=test-table [id=08fba0ce2fa948a196a791e7a6e7237b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:37.655432 20368 tablet_service.cc:1467] Processing CreateTablet for tablet 77705a72c4f94451bb000836277117cc (DEFAULT_TABLE table=test-table [id=08fba0ce2fa948a196a791e7a6e7237b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:37.656348 20368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77705a72c4f94451bb000836277117cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:37.656349 20231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77705a72c4f94451bb000836277117cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:37.660701 20505 tablet_service.cc:1467] Processing CreateTablet for tablet 77705a72c4f94451bb000836277117cc (DEFAULT_TABLE table=test-table [id=08fba0ce2fa948a196a791e7a6e7237b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:37.661842 20505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77705a72c4f94451bb000836277117cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:37.663316 20605 tablet_bootstrap.cc:492] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: Bootstrap starting.
I20260619 18:47:37.665642 20605 tablet_bootstrap.cc:654] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:37.666391 20605 log.cc:824] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:37.666545 20606 tablet_bootstrap.cc:492] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: Bootstrap starting.
I20260619 18:47:37.668083 20605 tablet_bootstrap.cc:492] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: No bootstrap required, opened a new log
I20260619 18:47:37.668268 20605 ts_tablet_manager.cc:1397] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 18:47:37.668758 20606 tablet_bootstrap.cc:654] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:37.669572 20606 log.cc:824] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:37.669601 20608 tablet_bootstrap.cc:492] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: Bootstrap starting.
I20260619 18:47:37.671581 20606 tablet_bootstrap.cc:492] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: No bootstrap required, opened a new log
I20260619 18:47:37.671774 20606 ts_tablet_manager.cc:1397] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 18:47:37.672149 20608 tablet_bootstrap.cc:654] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:37.672526 20605 raft_consensus.cc:348] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } }
I20260619 18:47:37.672847 20605 raft_consensus.cc:374] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:37.672905 20605 raft_consensus.cc:729] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e175e83617ab4b298c7abd17b946b6d2, State: Initialized, Role: FOLLOWER
I20260619 18:47:37.673000 20608 log.cc:824] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:37.673390 20605 consensus_queue.cc:260] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } }
I20260619 18:47:37.675009 20608 tablet_bootstrap.cc:492] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: No bootstrap required, opened a new log
I20260619 18:47:37.675204 20608 ts_tablet_manager.cc:1397] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 18:47:37.675316 20298 heartbeater.cc:503] Master 127.12.234.254:44503 was elected leader, sending a full tablet report...
I20260619 18:47:37.674400 20605 ts_tablet_manager.cc:1428] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260619 18:47:37.676014 20300 tablet.cc:2369] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:47:37.677012 20606 raft_consensus.cc:348] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } }
I20260619 18:47:37.677273 20606 raft_consensus.cc:374] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:37.677326 20606 raft_consensus.cc:729] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d6c6b4633cb43e2b95c1b40054cc3aa, State: Initialized, Role: FOLLOWER
I20260619 18:47:37.677795 20606 consensus_queue.cc:260] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } }
I20260619 18:47:37.678858 20435 heartbeater.cc:503] Master 127.12.234.254:44503 was elected leader, sending a full tablet report...
I20260619 18:47:37.680465 20606 ts_tablet_manager.cc:1428] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260619 18:47:37.681753 20608 raft_consensus.cc:348] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } }
I20260619 18:47:37.682058 20608 raft_consensus.cc:374] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:37.682125 20608 raft_consensus.cc:729] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff50c2d32a5412d8af73c53de93d3cf, State: Initialized, Role: FOLLOWER
I20260619 18:47:37.682621 20608 consensus_queue.cc:260] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } }
I20260619 18:47:37.683281 20572 heartbeater.cc:503] Master 127.12.234.254:44503 was elected leader, sending a full tablet report...
I20260619 18:47:37.683758 20608 ts_tablet_manager.cc:1428] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260619 18:47:37.760967 20613 raft_consensus.cc:482] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:37.761238 20613 raft_consensus.cc:504] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } }
I20260619 18:47:37.762497 20613 leader_election.cc:288] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e175e83617ab4b298c7abd17b946b6d2 (127.12.234.193:42443), 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539)
I20260619 18:47:37.766285 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.763009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:33843 (local address 127.12.234.194:38539)
0619 18:47:37.763135 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.763140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.763152 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:47:37.763205 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.763207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.763250 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.763348 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:37.763603 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.764178 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.765268 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.765686 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.765793 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.765847 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.766061 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.766134 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 18:47:37.766290 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.762747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:42443 (local address 127.12.234.195:43389)
0619 18:47:37.763013 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.763023 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.763138 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.763447 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.763450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.763463 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:37.763548 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.763555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.764310 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.764313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.765179 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.765189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.765793 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.765797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.765968 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.765986 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.766064 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.766134 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":219}
I20260619 18:47:37.766294 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.762885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:43389 (local address 127.12.234.193:42443)
0619 18:47:37.762998 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.763003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.763175 (+ 172us) server_negotiation.cc:407] Connection header received
0619 18:47:37.763234 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.763236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.763275 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.763341 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:37.763638 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.764198 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.765302 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.765690 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.765793 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.765847 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.766089 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.766160 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:47:37.766294 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.762747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:38539 (local address 127.12.234.195:33843)
0619 18:47:37.762940 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:37.762955 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:37.763049 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:37.763355 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:37.763358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:37.763374 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:37.763482 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.763488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.764277 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.764280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.765152 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:37.765162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.765794 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.765798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:37.765942 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.765971 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 18:47:37.766029 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 18:47:37.766102 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":26}
I20260619 18:47:37.766954 20251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77705a72c4f94451bb000836277117cc" candidate_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e175e83617ab4b298c7abd17b946b6d2" is_pre_election: true
I20260619 18:47:37.766987 20388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77705a72c4f94451bb000836277117cc" candidate_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" is_pre_election: true
I20260619 18:47:37.767244 20388 raft_consensus.cc:2454] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff50c2d32a5412d8af73c53de93d3cf in term 0.
I20260619 18:47:37.767246 20251 raft_consensus.cc:2454] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff50c2d32a5412d8af73c53de93d3cf in term 0.
I20260619 18:47:37.767681 20459 leader_election.cc:302] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff50c2d32a5412d8af73c53de93d3cf, 5d6c6b4633cb43e2b95c1b40054cc3aa; no voters:
I20260619 18:47:37.767939 20613 raft_consensus.cc:2781] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:37.768021 20613 raft_consensus.cc:482] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:37.768078 20613 raft_consensus.cc:3037] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:37.769066 20613 raft_consensus.cc:504] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } }
I20260619 18:47:37.769366 20613 leader_election.cc:288] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [CANDIDATE]: Term 1 election: Requested vote from peers e175e83617ab4b298c7abd17b946b6d2 (127.12.234.193:42443), 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539)
I20260619 18:47:37.769708 20251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77705a72c4f94451bb000836277117cc" candidate_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e175e83617ab4b298c7abd17b946b6d2"
I20260619 18:47:37.769698 20388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77705a72c4f94451bb000836277117cc" candidate_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa"
I20260619 18:47:37.769824 20388 raft_consensus.cc:3037] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:37.769824 20251 raft_consensus.cc:3037] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:37.771030 20388 raft_consensus.cc:2454] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff50c2d32a5412d8af73c53de93d3cf in term 1.
I20260619 18:47:37.771030 20251 raft_consensus.cc:2454] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff50c2d32a5412d8af73c53de93d3cf in term 1.
I20260619 18:47:37.771315 20460 leader_election.cc:302] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff50c2d32a5412d8af73c53de93d3cf, e175e83617ab4b298c7abd17b946b6d2; no voters:
I20260619 18:47:37.771476 20613 raft_consensus.cc:2781] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:37.771694 20613 raft_consensus.cc:686] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 1 LEADER]: Becoming Leader. State: Replica: 3ff50c2d32a5412d8af73c53de93d3cf, State: Running, Role: LEADER
I20260619 18:47:37.772043 20613 consensus_queue.cc:237] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } }
I20260619 18:47:37.775432 19334 catalog_manager.cc:5495] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ff50c2d32a5412d8af73c53de93d3cf (127.12.234.195). New cstate: current_term: 1 leader_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } health_report { overall_health: HEALTHY } } }
I20260619 18:47:37.801761 20604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.798080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.12.234.195:42317)
0619 18:47:37.798232 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.798237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.798251 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:37.798405 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.798407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.798466 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.798590 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:37.798843 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.799354 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.800284 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.800496 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.800629 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.800887 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:37.801359 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:37.801472 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.801535 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.801590 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 18:47:37.807981 20251 raft_consensus.cc:1261] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [term 1 FOLLOWER]: Refusing update from remote peer 3ff50c2d32a5412d8af73c53de93d3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:37.807981 20388 raft_consensus.cc:1261] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 1 FOLLOWER]: Refusing update from remote peer 3ff50c2d32a5412d8af73c53de93d3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:37.808821 20614 consensus_queue.cc:1035] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:37.809087 20613 consensus_queue.cc:1035] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:37.816919 20617 mvcc.cc:205] Tried to move back new op lower bound from 7298641337573253120 to 7298641337437151232. Current Snapshot: MvccSnapshot[applied={T|T < 7298641337573253120}]
I20260619 18:47:37.824141 20619 mvcc.cc:205] Tried to move back new op lower bound from 7298641337573253120 to 7298641337437151232. Current Snapshot: MvccSnapshot[applied={T|T < 7298641337573253120}]
I20260619 18:47:37.833650 19333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:37.833846 19333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:37.838289 19333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfa9f9d9741744ca8c9985b264c1d108: Sending DeleteTablet for 3 replicas of tablet 77705a72c4f94451bb000836277117cc
I20260619 18:47:37.839040 20231 tablet_service.cc:1514] Processing DeleteTablet for tablet 77705a72c4f94451bb000836277117cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:37 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:46656
I20260619 18:47:37.839552 20626 tablet_replica.cc:330] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: stopping tablet replica
I20260619 18:47:37.839551 20368 tablet_service.cc:1514] Processing DeleteTablet for tablet 77705a72c4f94451bb000836277117cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:37 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47380
I20260619 18:47:37.839915 20626 raft_consensus.cc:2229] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:37.840090 20627 tablet_replica.cc:330] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: stopping tablet replica
I20260619 18:47:37.840294 20626 raft_consensus.cc:2258] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:37.840391 20505 tablet_service.cc:1514] Processing DeleteTablet for tablet 77705a72c4f94451bb000836277117cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:37 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:55020
I20260619 18:47:37.840490 20627 raft_consensus.cc:2229] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:37.840854 20627 raft_consensus.cc:2258] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:37.842053 20626 ts_tablet_manager.cc:1905] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:37.842522 20627 ts_tablet_manager.cc:1905] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:37.843700 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 19302
I20260619 18:47:37.845906 20626 ts_tablet_manager.cc:1918] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:37.846004 20626 log.cc:1196] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/77705a72c4f94451bb000836277117cc
I20260619 18:47:37.846591 20626 ts_tablet_manager.cc:1939] T 77705a72c4f94451bb000836277117cc P e175e83617ab4b298c7abd17b946b6d2: Deleting consensus metadata
I20260619 18:47:37.847138 20627 ts_tablet_manager.cc:1918] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:37.847293 20627 log.cc:1196] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/77705a72c4f94451bb000836277117cc
I20260619 18:47:37.847644 20627 ts_tablet_manager.cc:1939] T 77705a72c4f94451bb000836277117cc P 5d6c6b4633cb43e2b95c1b40054cc3aa: Deleting consensus metadata
I20260619 18:47:37.848524 20628 tablet_replica.cc:330] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: stopping tablet replica
I20260619 18:47:37.848866 20628 raft_consensus.cc:2229] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:37.849345 20628 raft_consensus.cc:2258] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:37.851087 20628 ts_tablet_manager.cc:1905] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 18:47:37.851133 20185 connection.cc:535] server connection from 127.0.0.1:46656 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 18:47:37.854100 20628 ts_tablet_manager.cc:1918] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:37.854231 20628 log.cc:1196] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/77705a72c4f94451bb000836277117cc
I20260619 18:47:37.854538 20628 ts_tablet_manager.cc:1939] T 77705a72c4f94451bb000836277117cc P 3ff50c2d32a5412d8af73c53de93d3cf: Deleting consensus metadata
W20260619 18:47:37.855110 20321 connection.cc:441] server connection from 127.0.0.1:47380 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47380 (request call id 1) could send its response
W20260619 18:47:37.855652 20459 connection.cc:441] server connection from 127.0.0.1:55020 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55020 (request call id 1) could send its response
I20260619 18:47:37.862876 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.842741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48682 (local address 127.12.234.253:36807)
0619 18:47:37.842858 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.842862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.848228 (+ 5366us) server_negotiation.cc:407] Connection header received
0619 18:47:37.848365 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.848368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.848426 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.848497 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.849708 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.850475 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.851688 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.852036 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.852117 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.855201 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.855205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.855208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.855242 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.856973 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.857654 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.857658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.857660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.857753 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.858030 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.858033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.858035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.858289 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.858417 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.862563 (+ 4146us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.862703 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 18:47:37.866533 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.856201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48688 (local address 127.12.234.253:36807)
0619 18:47:37.856314 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.856319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.856549 (+ 230us) server_negotiation.cc:407] Connection header received
0619 18:47:37.856737 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.856741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.856789 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.856838 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.857679 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.858509 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.859754 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.860125 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.860221 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.862242 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.862245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.862249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.862274 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.864160 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.864929 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.864933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.864935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.865019 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.865534 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.865538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.865539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.865764 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.865865 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.866262 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.866375 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:47:37.868254 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.857215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46560 (local address 127.12.234.252:37131)
0619 18:47:37.857331 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.857335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.857551 (+ 216us) server_negotiation.cc:407] Connection header received
0619 18:47:37.857714 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.857717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.857766 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.857818 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.858636 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.859351 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.861588 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.861922 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.862001 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.864281 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.864284 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.864287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.864312 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.866000 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.866518 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.866521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.866523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.866613 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.867372 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.867375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.867377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.867581 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.867684 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.867996 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.868106 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 18:47:37.868386 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:37.842514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46554 (local address 127.12.234.252:37131)
0619 18:47:37.842633 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:37.842638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:37.859195 (+ 16557us) server_negotiation.cc:407] Connection header received
0619 18:47:37.859237 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:37.859240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:37.859284 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:37.859343 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:37.860124 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.861150 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.862158 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:37.862896 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:37.862991 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:37.865165 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:37.865169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:37.865172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:37.865196 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:37.866821 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.867186 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.867189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.867191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.867262 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:37.867527 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:37.867530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:37.867531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:37.867695 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:37.867797 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:37.868077 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 18:47:37.868202 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:47:38.821676 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:38.821163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.194:52563)
0619 18:47:38.821417 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:38.821540 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260619 18:47:38.821664 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:38.821264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:49761)
0619 18:47:38.821514 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:38.821586 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":56,"threads_started":1}
I20260619 18:47:38.821697 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:38.821166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.195:50037)
0619 18:47:38.821493 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:38.821577 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":77,"threads_started":1}
W20260619 18:47:38.822002 20572 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:44503 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
W20260619 18:47:38.822310 20435 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:44503 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
W20260619 18:47:38.822583 20298 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:44503 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
I20260619 18:47:39.338564 20640 raft_consensus.cc:482] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bfa9f9d9741744ca8c9985b264c1d108)
I20260619 18:47:39.338754 20640 raft_consensus.cc:504] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } }
I20260619 18:47:39.339063 20641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bfa9f9d9741744ca8c9985b264c1d108)
I20260619 18:47:39.339236 20641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } }
I20260619 18:47:39.339852 20640 leader_election.cc:288] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503), 6e128141cfe0477fbc60a094851bd867 (127.12.234.252:37131)
I20260619 18:47:39.340139 20641 leader_election.cc:288] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503), c675f9ddf30346499d349179d2668d8a (127.12.234.253:36807)
I20260619 18:47:39.340756 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.340014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52552)
0619 18:47:39.340565 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.340655 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":67,"threads_started":1}
I20260619 18:47:39.340808 20644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.340243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52558)
0619 18:47:39.340637 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.340715 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":63,"threads_started":1}
W20260619 18:47:39.340929 19920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
W20260619 18:47:39.340957 20103 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
I20260619 18:47:39.341816 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.341601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52564)
0619 18:47:39.341722 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.341759 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 18:47:39.341816 20644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.341587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52562)
0619 18:47:39.341701 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.341751 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 18:47:39.341912 19920 leader_election.cc:334] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503): Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
W20260619 18:47:39.342008 20103 leader_election.cc:334] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503): Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
I20260619 18:47:39.345446 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.340221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55458 (local address 127.12.234.252:37131)
0619 18:47:39.340615 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:39.340620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:39.340939 (+ 319us) server_negotiation.cc:407] Connection header received
0619 18:47:39.341176 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:39.341179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:39.341224 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:39.341293 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:39.341539 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:39.342855 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:39.344306 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:39.344914 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:39.345029 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:39.345081 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:39.345282 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 18:47:39.345349 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":348,"thread_start_us":68,"threads_started":1}
I20260619 18:47:39.345443 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.340397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:37131 (local address 127.0.0.1:55458)
0619 18:47:39.340672 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.340683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:39.340787 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:39.341345 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:39.341348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:39.341364 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:39.341453 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:39.341460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:39.342965 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:39.342968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:39.344186 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:39.344197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:39.345016 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:39.345019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:39.345110 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:39.345136 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 18:47:39.345228 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 18:47:39.345303 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":54,"threads_started":1}
I20260619 18:47:39.345450 20647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.340402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47030 (local address 127.12.234.253:36807)
0619 18:47:39.340580 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:39.340584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:39.340783 (+ 199us) server_negotiation.cc:407] Connection header received
0619 18:47:39.340976 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:39.340980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:39.341024 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:39.341078 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:39.342143 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:39.342776 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:39.344306 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:39.344914 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:39.345029 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:39.345085 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:39.345296 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 18:47:39.345358 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":43,"threads_started":1}
I20260619 18:47:39.345829 20645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.340332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:36807 (local address 127.0.0.1:47030)
0619 18:47:39.340710 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.340720 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:39.340856 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:39.341239 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:39.341242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:39.341259 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:39.341381 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:39.341388 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:39.342870 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:39.342873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:39.344186 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:39.344197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:39.345016 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:39.345019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:39.345154 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:39.345176 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:47:39.345469 (+ 293us) client_negotiation.cc:240] Negotiation successful
0619 18:47:39.345717 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":2,"thread_start_us":45,"threads_started":1}
I20260619 18:47:39.345996 20124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c675f9ddf30346499d349179d2668d8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6e128141cfe0477fbc60a094851bd867" is_pre_election: true
I20260619 18:47:39.346263 20124 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c675f9ddf30346499d349179d2668d8a in term 1.
I20260619 18:47:39.346261 19945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e128141cfe0477fbc60a094851bd867" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c675f9ddf30346499d349179d2668d8a" is_pre_election: true
I20260619 18:47:39.346464 19945 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e128141cfe0477fbc60a094851bd867 in term 1.
I20260619 18:47:39.346635 19919 leader_election.cc:302] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e128141cfe0477fbc60a094851bd867, c675f9ddf30346499d349179d2668d8a; no voters: bfa9f9d9741744ca8c9985b264c1d108
I20260619 18:47:39.346828 20640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 18:47:39.346804 20102 leader_election.cc:302] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e128141cfe0477fbc60a094851bd867, c675f9ddf30346499d349179d2668d8a; no voters: bfa9f9d9741744ca8c9985b264c1d108
I20260619 18:47:39.346872 20640 raft_consensus.cc:482] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader bfa9f9d9741744ca8c9985b264c1d108)
I20260619 18:47:39.346907 20640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 1 FOLLOWER]: Advancing to term 2
I20260619 18:47:39.347141 20641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 18:47:39.347196 20641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bfa9f9d9741744ca8c9985b264c1d108)
I20260619 18:47:39.347240 20641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 1 FOLLOWER]: Advancing to term 2
I20260619 18:47:39.348214 20640 raft_consensus.cc:504] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } }
I20260619 18:47:39.348336 20641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } }
I20260619 18:47:39.348618 20640 leader_election.cc:288] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 2 election: Requested vote from peers bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503), 6e128141cfe0477fbc60a094851bd867 (127.12.234.252:37131)
I20260619 18:47:39.348666 20641 leader_election.cc:288] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [CANDIDATE]: Term 2 election: Requested vote from peers bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503), c675f9ddf30346499d349179d2668d8a (127.12.234.253:36807)
I20260619 18:47:39.349004 20645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.348742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52576)
0619 18:47:39.348890 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.348941 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 18:47:39.349004 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.348738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52572)
0619 18:47:39.348884 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.348937 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 18:47:39.349016 19945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e128141cfe0477fbc60a094851bd867" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c675f9ddf30346499d349179d2668d8a"
I20260619 18:47:39.349179 19945 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e128141cfe0477fbc60a094851bd867 in current term 2: Already voted for candidate c675f9ddf30346499d349179d2668d8a in this term.
I20260619 18:47:39.349275 20124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c675f9ddf30346499d349179d2668d8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6e128141cfe0477fbc60a094851bd867"
I20260619 18:47:39.349431 20124 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c675f9ddf30346499d349179d2668d8a in current term 2: Already voted for candidate 6e128141cfe0477fbc60a094851bd867 in this term.
I20260619 18:47:39.349617 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.349395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52578)
0619 18:47:39.349496 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.349546 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 18:47:39.349692 19920 leader_election.cc:334] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503): Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
I20260619 18:47:39.349772 19919 leader_election.cc:302] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c675f9ddf30346499d349179d2668d8a; no voters: 6e128141cfe0477fbc60a094851bd867, bfa9f9d9741744ca8c9985b264c1d108
I20260619 18:47:39.349905 20640 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 18:47:39.350060 20645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.349675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52582)
0619 18:47:39.349952 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.350003 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
W20260619 18:47:39.350171 20103 leader_election.cc:334] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503): Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
I20260619 18:47:39.350299 20103 leader_election.cc:302] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e128141cfe0477fbc60a094851bd867; no voters: bfa9f9d9741744ca8c9985b264c1d108, c675f9ddf30346499d349179d2668d8a
I20260619 18:47:39.350416 20641 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 18:47:39.823414 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.822878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.194:36119)
0619 18:47:39.823154 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.823259 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":79,"threads_started":1}
I20260619 18:47:39.823474 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.822748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.195:42889)
0619 18:47:39.823314 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.823391 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":83,"threads_started":1}
I20260619 18:47:39.823693 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:39.823161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:54581)
0619 18:47:39.823445 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:39.823583 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":67,"threads_started":1}
I20260619 18:47:40.825022 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:40.824393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.195:56155)
0619 18:47:40.824785 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:40.824861 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":72,"threads_started":1}
I20260619 18:47:40.825093 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:40.824456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:33103)
0619 18:47:40.824922 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:40.824992 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":67,"threads_started":1}
I20260619 18:47:40.825317 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:40.824485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.194:47365)
0619 18:47:40.825055 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:40.825116 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":52,"threads_started":1}
I20260619 18:47:41.015208 20656 raft_consensus.cc:482] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:41.015352 20656 raft_consensus.cc:504] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } }
I20260619 18:47:41.015821 20656 leader_election.cc:288] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503), 6e128141cfe0477fbc60a094851bd867 (127.12.234.252:37131)
I20260619 18:47:41.016364 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.015989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52838)
0619 18:47:41.016208 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.016273 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":66,"threads_started":1}
I20260619 18:47:41.016316 20124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c675f9ddf30346499d349179d2668d8a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6e128141cfe0477fbc60a094851bd867" is_pre_election: true
I20260619 18:47:41.016467 20124 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c675f9ddf30346499d349179d2668d8a in term 2.
I20260619 18:47:41.016832 19919 leader_election.cc:302] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e128141cfe0477fbc60a094851bd867, c675f9ddf30346499d349179d2668d8a; no voters:
I20260619 18:47:41.017007 20656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 18:47:41.017067 20656 raft_consensus.cc:482] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:41.017110 20656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 2 FOLLOWER]: Advancing to term 3
I20260619 18:47:41.017186 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.016952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52854)
0619 18:47:41.017064 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.017120 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 18:47:41.017277 19920 leader_election.cc:334] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503): Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
I20260619 18:47:41.018225 20656 raft_consensus.cc:504] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } }
I20260619 18:47:41.018591 20656 leader_election.cc:288] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 3 election: Requested vote from peers bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503), 6e128141cfe0477fbc60a094851bd867 (127.12.234.252:37131)
I20260619 18:47:41.018867 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.018691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52858)
0619 18:47:41.018774 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.018817 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 18:47:41.018994 20124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c675f9ddf30346499d349179d2668d8a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6e128141cfe0477fbc60a094851bd867"
I20260619 18:47:41.019116 20124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 2 FOLLOWER]: Advancing to term 3
I20260619 18:47:41.019493 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.019294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52870)
0619 18:47:41.019392 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.019439 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 18:47:41.019618 19920 leader_election.cc:334] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503): Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
I20260619 18:47:41.020170 20124 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c675f9ddf30346499d349179d2668d8a in term 3.
I20260619 18:47:41.020445 19919 leader_election.cc:302] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e128141cfe0477fbc60a094851bd867, c675f9ddf30346499d349179d2668d8a; no voters: bfa9f9d9741744ca8c9985b264c1d108
I20260619 18:47:41.020597 20656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 3 FOLLOWER]: Leader election won for term 3
I20260619 18:47:41.020779 20656 raft_consensus.cc:686] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [term 3 LEADER]: Becoming Leader. State: Replica: c675f9ddf30346499d349179d2668d8a, State: Running, Role: LEADER
I20260619 18:47:41.021044 20656 consensus_queue.cc:237] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } }
I20260619 18:47:41.022580 20659 sys_catalog.cc:455] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c675f9ddf30346499d349179d2668d8a. Latest consensus state: current_term: 3 leader_uuid: "c675f9ddf30346499d349179d2668d8a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:41.022683 20659 sys_catalog.cc:458] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:41.022969 20661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:41.025317 20661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:41.025697 20661 catalog_manager.cc:1259] Loaded cluster ID: dff4f39ee9de43c7843284c70fc51efe
I20260619 18:47:41.025743 20661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:41.026293 20661 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:41.026655 20661 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Loaded TSK: 0
I20260619 18:47:41.027406 20661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:47:41.081897 19933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:41.084909 19933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:41.087903 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.087639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52888)
0619 18:47:41.087755 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.087821 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 18:47:41.087963 20124 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [term 3 FOLLOWER]: Refusing update from remote peer c675f9ddf30346499d349179d2668d8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 18:47:41.088426 20656 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 18:47:41.088563 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.088368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.0.0.1:52892)
0619 18:47:41.088463 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.088509 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 18:47:41.088662 19920 consensus_peers.cc:487] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a -> Peer bfa9f9d9741744ca8c9985b264c1d108 (127.12.234.254:44503): Couldn't send request to peer bfa9f9d9741744ca8c9985b264c1d108. Status: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 18:47:41.089845 20663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c675f9ddf30346499d349179d2668d8a. Latest consensus state: current_term: 3 leader_uuid: "c675f9ddf30346499d349179d2668d8a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:41.089960 20663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:47:41.091101 20656 sys_catalog.cc:455] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c675f9ddf30346499d349179d2668d8a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:41.091228 20656 sys_catalog.cc:458] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:41.091701 20663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c675f9ddf30346499d349179d2668d8a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9f9d9741744ca8c9985b264c1d108" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44503 } } peers { permanent_uuid: "c675f9ddf30346499d349179d2668d8a" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 36807 } attrs { promote: false } } peers { permanent_uuid: "6e128141cfe0477fbc60a094851bd867" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 37131 } attrs { promote: false } } }
I20260619 18:47:41.091822 20663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e128141cfe0477fbc60a094851bd867 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:47:41.101594 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.097489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:42443 (local address 127.0.0.1:57258)
0619 18:47:41.097831 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.097844 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:41.097940 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:41.098321 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:41.098324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:41.098340 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:41.098435 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.098441 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.099443 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.099446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.100426 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.100435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.101179 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.101182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.101264 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.101291 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 18:47:41.101353 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 18:47:41.101425 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":23,"thread_start_us":100,"threads_started":1}
I20260619 18:47:41.102017 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.097166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.12.234.195:42317)
0619 18:47:41.098209 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:41.098213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:41.098227 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:41.098273 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:41.098276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:41.098320 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:41.098748 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:41.098860 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.099600 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.100944 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.101517 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.101626 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.101771 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:41.101831 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 18:47:41.101903 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933,"thread_start_us":72,"threads_started":1}
I20260619 18:47:41.102013 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.097038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:42317 (local address 127.0.0.1:52956)
0619 18:47:41.097193 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.097209 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:41.097297 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:41.098430 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:41.098433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:41.098445 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:41.098544 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.098550 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.099713 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.099716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.100842 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.100851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.101632 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.101635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.101720 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.101744 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 18:47:41.101813 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 18:47:41.101881 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":4}
I20260619 18:47:41.102030 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.097600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.12.234.193:42443)
0619 18:47:41.097830 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:41.097835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:41.097905 (+ 70us) server_negotiation.cc:407] Connection header received
0619 18:47:41.098040 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:41.098043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:41.098093 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:41.098210 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:41.098569 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.099325 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.100531 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.101083 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.101199 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.101251 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:41.101811 (+ 560us) server_negotiation.cc:299] Negotiation successful
0619 18:47:41.101901 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":66,"threads_started":1}
I20260619 18:47:41.102524 20231 tablet_service.cc:1467] Processing CreateTablet for tablet be57a33e7ad84e0b9c145102c42eb337 (DEFAULT_TABLE table=test-table [id=50d33a926db34fca9f773fb104ea8778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:41.102624 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.097263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:38539 (local address 127.0.0.1:52208)
0619 18:47:41.097491 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.097503 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:41.097577 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:41.098106 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:41.098110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:41.098125 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:41.098263 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.098270 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.099288 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.099292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.100474 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.100484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.101780 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.101784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.101867 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.101889 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:47:41.101933 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 18:47:41.101997 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":49,"thread_start_us":109,"threads_started":1}
I20260619 18:47:41.102828 20231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be57a33e7ad84e0b9c145102c42eb337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:41.104987 20673 tablet_bootstrap.cc:492] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2: Bootstrap starting.
I20260619 18:47:41.105192 20505 tablet_service.cc:1467] Processing CreateTablet for tablet be57a33e7ad84e0b9c145102c42eb337 (DEFAULT_TABLE table=test-table [id=50d33a926db34fca9f773fb104ea8778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:41.105418 20505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be57a33e7ad84e0b9c145102c42eb337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:41.105886 20673 tablet_bootstrap.cc:654] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:41.107002 20673 tablet_bootstrap.cc:492] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2: No bootstrap required, opened a new log
I20260619 18:47:41.107085 20673 ts_tablet_manager.cc:1397] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 18:47:41.107466 20675 tablet_bootstrap.cc:492] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: Bootstrap starting.
I20260619 18:47:41.107502 20673 raft_consensus.cc:348] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } }
I20260619 18:47:41.107613 20673 raft_consensus.cc:374] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:41.107657 20673 raft_consensus.cc:729] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e175e83617ab4b298c7abd17b946b6d2, State: Initialized, Role: FOLLOWER
I20260619 18:47:41.107812 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.097616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52208 (local address 127.12.234.194:38539)
0619 18:47:41.097881 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:41.097886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:41.097900 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:41.097949 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:41.097952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:41.097998 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:41.098367 (+ 369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:41.098474 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.099193 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.100605 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.101656 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.107451 (+ 5795us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.107533 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:41.107577 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 18:47:41.107669 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
I20260619 18:47:41.107811 20673 consensus_queue.cc:260] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } }
I20260619 18:47:41.108286 20368 tablet_service.cc:1467] Processing CreateTablet for tablet be57a33e7ad84e0b9c145102c42eb337 (DEFAULT_TABLE table=test-table [id=50d33a926db34fca9f773fb104ea8778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:41.108466 20673 ts_tablet_manager.cc:1428] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 18:47:41.108560 20368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be57a33e7ad84e0b9c145102c42eb337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:41.108773 20675 tablet_bootstrap.cc:654] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:41.109498 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.109266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:43333)
0619 18:47:41.109371 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.109427 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 18:47:41.110136 20676 raft_consensus.cc:482] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:41.110248 20676 raft_consensus.cc:504] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } }
I20260619 18:47:41.110375 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.110104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:51547)
0619 18:47:41.110269 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.110317 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260619 18:47:41.111048 20678 tablet_bootstrap.cc:492] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: Bootstrap starting.
I20260619 18:47:41.111331 20676 leader_election.cc:288] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ff50c2d32a5412d8af73c53de93d3cf (127.12.234.195:42317), 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539)
I20260619 18:47:41.112041 20675 tablet_bootstrap.cc:492] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: No bootstrap required, opened a new log
I20260619 18:47:41.112112 20299 heartbeater.cc:503] Master 127.12.234.253:36807 was elected leader, sending a full tablet report...
I20260619 18:47:41.112138 20675 ts_tablet_manager.cc:1397] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 18:47:41.112560 20678 tablet_bootstrap.cc:654] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:41.112540 20675 raft_consensus.cc:348] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } }
I20260619 18:47:41.112654 20675 raft_consensus.cc:374] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:41.112687 20675 raft_consensus.cc:729] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff50c2d32a5412d8af73c53de93d3cf, State: Initialized, Role: FOLLOWER
I20260619 18:47:41.112808 20675 consensus_queue.cc:260] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } }
I20260619 18:47:41.113138 20675 ts_tablet_manager.cc:1428] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:41.113871 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.113602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.195:39997)
0619 18:47:41.113750 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.113803 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 18:47:41.114655 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.114415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.195:50069)
0619 18:47:41.114539 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.114584 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 18:47:41.115533 20573 heartbeater.cc:503] Master 127.12.234.253:36807 was elected leader, sending a full tablet report...
I20260619 18:47:41.116058 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.111475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:42317 (local address 127.12.234.193:35857)
0619 18:47:41.111571 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.111585 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:41.111676 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:41.112331 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:41.112334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:41.112351 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:41.112477 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.112485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.113441 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.113444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.114583 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.114594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.115691 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.115694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.115793 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.115818 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 18:47:41.115876 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 18:47:41.115944 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 18:47:41.116621 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.112007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:35857 (local address 127.12.234.195:42317)
0619 18:47:41.112114 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:41.112129 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:41.112143 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:41.112186 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:41.112188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:41.112230 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:41.112305 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:41.112621 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.113289 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.114848 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.115399 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.115513 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.115574 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:41.116401 (+ 827us) server_negotiation.cc:299] Negotiation successful
0619 18:47:41.116473 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":31}
I20260619 18:47:41.117069 20525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be57a33e7ad84e0b9c145102c42eb337" candidate_uuid: "e175e83617ab4b298c7abd17b946b6d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" is_pre_election: true
I20260619 18:47:41.117254 20525 raft_consensus.cc:2454] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e175e83617ab4b298c7abd17b946b6d2 in term 0.
I20260619 18:47:41.117626 20185 leader_election.cc:302] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff50c2d32a5412d8af73c53de93d3cf, e175e83617ab4b298c7abd17b946b6d2; no voters:
I20260619 18:47:41.117892 20678 tablet_bootstrap.cc:492] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: No bootstrap required, opened a new log
I20260619 18:47:41.117895 20676 raft_consensus.cc:2781] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:41.117945 20676 raft_consensus.cc:482] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:41.117988 20678 ts_tablet_manager.cc:1397] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260619 18:47:41.117987 20676 raft_consensus.cc:3037] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:41.118469 20678 raft_consensus.cc:348] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } }
I20260619 18:47:41.118588 20678 raft_consensus.cc:374] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:41.118618 20678 raft_consensus.cc:729] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d6c6b4633cb43e2b95c1b40054cc3aa, State: Initialized, Role: FOLLOWER
I20260619 18:47:41.118741 20678 consensus_queue.cc:260] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } }
I20260619 18:47:41.119098 20678 ts_tablet_manager.cc:1428] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:41.119181 20679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.111700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:38539 (local address 127.12.234.193:56161)
0619 18:47:41.115285 (+ 3585us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.115298 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:41.115518 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:41.115847 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:41.115849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:41.115865 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:41.115961 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.115966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.116913 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.116916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.118061 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:41.118071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.118768 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.118772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:41.118869 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.118892 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 18:47:41.118959 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 18:47:41.119025 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3525,"thread_start_us":87,"threads_started":1}
I20260619 18:47:41.119207 20676 raft_consensus.cc:504] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } }
I20260619 18:47:41.119395 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.113867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:56161 (local address 127.12.234.194:38539)
0619 18:47:41.115142 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:41.115146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:41.115360 (+ 214us) server_negotiation.cc:407] Connection header received
0619 18:47:41.115636 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:41.115639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:41.115685 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:41.115729 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:41.116099 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.116780 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.118198 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.118602 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.118730 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.118791 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:41.118974 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 18:47:41.119044 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202}
I20260619 18:47:41.119534 20676 leader_election.cc:288] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [CANDIDATE]: Term 1 election: Requested vote from peers 3ff50c2d32a5412d8af73c53de93d3cf (127.12.234.195:42317), 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539)
I20260619 18:47:41.120245 20388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be57a33e7ad84e0b9c145102c42eb337" candidate_uuid: "e175e83617ab4b298c7abd17b946b6d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" is_pre_election: true
I20260619 18:47:41.120355 20388 raft_consensus.cc:2454] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e175e83617ab4b298c7abd17b946b6d2 in term 0.
I20260619 18:47:41.120833 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.120625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.194:45063)
0619 18:47:41.120724 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.120775 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 18:47:41.121500 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.121306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.194:41247)
0619 18:47:41.121400 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.121446 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 18:47:41.121685 20525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be57a33e7ad84e0b9c145102c42eb337" candidate_uuid: "e175e83617ab4b298c7abd17b946b6d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ff50c2d32a5412d8af73c53de93d3cf"
I20260619 18:47:41.121747 20388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be57a33e7ad84e0b9c145102c42eb337" candidate_uuid: "e175e83617ab4b298c7abd17b946b6d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa"
I20260619 18:47:41.121784 20525 raft_consensus.cc:3037] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:41.121843 20388 raft_consensus.cc:3037] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:41.122738 20388 raft_consensus.cc:2454] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e175e83617ab4b298c7abd17b946b6d2 in term 1.
I20260619 18:47:41.122749 20525 raft_consensus.cc:2454] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e175e83617ab4b298c7abd17b946b6d2 in term 1.
I20260619 18:47:41.122980 20185 leader_election.cc:302] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff50c2d32a5412d8af73c53de93d3cf, e175e83617ab4b298c7abd17b946b6d2; no voters:
I20260619 18:47:41.123169 20676 raft_consensus.cc:2781] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:41.123379 20676 raft_consensus.cc:686] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [term 1 LEADER]: Becoming Leader. State: Replica: e175e83617ab4b298c7abd17b946b6d2, State: Running, Role: LEADER
I20260619 18:47:41.123629 20676 consensus_queue.cc:237] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } }
I20260619 18:47:41.125190 20679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.124889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:35265)
0619 18:47:41.125069 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.125120 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260619 18:47:41.125952 19933 catalog_manager.cc:5495] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e175e83617ab4b298c7abd17b946b6d2 (127.12.234.193). New cstate: current_term: 1 leader_uuid: "e175e83617ab4b298c7abd17b946b6d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e175e83617ab4b298c7abd17b946b6d2" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42443 } health_report { overall_health: HEALTHY } } }
W20260619 18:47:41.135682 20437 tablet.cc:2369] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:47:41.145217 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.141162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57274 (local address 127.12.234.193:42443)
0619 18:47:41.141281 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:41.141286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:41.141319 (+ 33us) server_negotiation.cc:407] Connection header received
0619 18:47:41.141552 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:41.141555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:41.141624 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:41.141722 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:41.141950 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.142709 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.143786 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:41.144025 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:41.144111 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:41.144366 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:41.144842 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:41.144973 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:41.145015 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 18:47:41.145076 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 18:47:41.148090 20388 raft_consensus.cc:1261] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 1 FOLLOWER]: Refusing update from remote peer e175e83617ab4b298c7abd17b946b6d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:41.148355 20525 raft_consensus.cc:1261] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [term 1 FOLLOWER]: Refusing update from remote peer e175e83617ab4b298c7abd17b946b6d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:41.148717 20683 consensus_queue.cc:1035] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d6c6b4633cb43e2b95c1b40054cc3aa" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:41.148890 20676 consensus_queue.cc:1035] T be57a33e7ad84e0b9c145102c42eb337 P e175e83617ab4b298c7abd17b946b6d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ff50c2d32a5412d8af73c53de93d3cf" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:41.150244 20436 heartbeater.cc:503] Master 127.12.234.253:36807 was elected leader, sending a full tablet report...
I20260619 18:47:41.150810 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.150521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.194:42193)
0619 18:47:41.150675 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.150733 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260619 18:47:41.150810 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.150521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.195:54567)
0619 18:47:41.150675 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.150733 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260619 18:47:41.153086 20687 mvcc.cc:205] Tried to move back new op lower bound from 7298641351257702400 to 7298641351165067264. Current Snapshot: MvccSnapshot[applied={T|T < 7298641351257702400}]
I20260619 18:47:41.154840 20679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.154498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:57707)
0619 18:47:41.154696 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.154761 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260619 18:47:41.155004 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.154776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.195:51691)
0619 18:47:41.154881 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.154932 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 18:47:41.162084 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.161726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.194:55961)
0619 18:47:41.161906 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.161999 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260619 18:47:41.162077 19933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:41.162266 19933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:41.162755 20679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.162505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44503 (local address 127.12.234.193:48703)
0619 18:47:41.162618 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.162680 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 18:47:41.166769 19933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c675f9ddf30346499d349179d2668d8a: Sending DeleteTablet for 3 replicas of tablet be57a33e7ad84e0b9c145102c42eb337
I20260619 18:47:41.167559 20367 tablet_service.cc:1514] Processing DeleteTablet for tablet be57a33e7ad84e0b9c145102c42eb337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:41 UTC) from {username='oryx', principal='oryx/127.12.234.253@KRBTEST.COM'} at 127.0.0.1:52208
I20260619 18:47:41.167585 20505 tablet_service.cc:1514] Processing DeleteTablet for tablet be57a33e7ad84e0b9c145102c42eb337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:41 UTC) from {username='oryx', principal='oryx/127.12.234.253@KRBTEST.COM'} at 127.0.0.1:52956
I20260619 18:47:41.167892 20368 tablet_service.cc:1514] Processing DeleteTablet for tablet be57a33e7ad84e0b9c145102c42eb337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:41 UTC) from {username='oryx', principal='oryx/127.12.234.253@KRBTEST.COM'} at 127.0.0.1:52208
I20260619 18:47:41.168081 20231 tablet_service.cc:1514] Processing DeleteTablet for tablet be57a33e7ad84e0b9c145102c42eb337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:41 UTC) from {username='oryx', principal='oryx/127.12.234.253@KRBTEST.COM'} at 127.0.0.1:57258
I20260619 18:47:41.168174 20698 tablet_replica.cc:330] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: stopping tablet replica
I20260619 18:47:41.168324 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 20080
I20260619 18:47:41.169780 20698 raft_consensus.cc:2229] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:41.169945 20698 raft_consensus.cc:2258] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:41.170292 20697 tablet_replica.cc:330] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: stopping tablet replica
I20260619 18:47:41.170413 20697 raft_consensus.cc:2229] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:41.170529 20697 raft_consensus.cc:2258] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:41.170984 20698 ts_tablet_manager.cc:1905] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:41.171144 20697 ts_tablet_manager.cc:1905] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:41.172986 20698 ts_tablet_manager.cc:1918] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:41.173058 20698 log.cc:1196] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/be57a33e7ad84e0b9c145102c42eb337
I20260619 18:47:41.173032 20697 ts_tablet_manager.cc:1918] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:41.173101 20697 log.cc:1196] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/be57a33e7ad84e0b9c145102c42eb337
I20260619 18:47:41.173312 20698 ts_tablet_manager.cc:1939] T be57a33e7ad84e0b9c145102c42eb337 P 3ff50c2d32a5412d8af73c53de93d3cf: Deleting consensus metadata
I20260619 18:47:41.173323 20697 ts_tablet_manager.cc:1939] T be57a33e7ad84e0b9c145102c42eb337 P 5d6c6b4633cb43e2b95c1b40054cc3aa: Deleting consensus metadata
I20260619 18:47:41.174299 19920 catalog_manager.cc:4841] TS 3ff50c2d32a5412d8af73c53de93d3cf (127.12.234.195:42317): tablet be57a33e7ad84e0b9c145102c42eb337 (table test-table [id=50d33a926db34fca9f773fb104ea8778]) successfully deleted
I20260619 18:47:41.174530 19920 catalog_manager.cc:4841] TS 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539): tablet be57a33e7ad84e0b9c145102c42eb337 (table test-table [id=50d33a926db34fca9f773fb104ea8778]) successfully deleted
W20260619 18:47:41.174772 19920 catalog_manager.cc:4808] TS 5d6c6b4633cb43e2b95c1b40054cc3aa (127.12.234.194:38539): delete failed for tablet be57a33e7ad84e0b9c145102c42eb337 because the tablet was not found. No further retry: Not found: Tablet not found: be57a33e7ad84e0b9c145102c42eb337
I20260619 18:47:41.177709 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 20305
I20260619 18:47:41.177963 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:41.177693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:42443 (local address 127.0.0.1:57282)
0619 18:47:41.177828 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:41.177882 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:42443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 18:47:41.178350 19921 catalog_manager.cc:4584] TS e175e83617ab4b298c7abd17b946b6d2 (127.12.234.193:42443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be57a33e7ad84e0b9c145102c42eb337: Network error: Client connection negotiation failed: client connection to 127.12.234.193:42443: connect: Connection refused (error 111)
I20260619 18:47:41.185091 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 20441
I20260619 18:47:41.191002 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 19798
I20260619 18:47:41.195941 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 19900
2026-06-19T18:47:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6889 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:41 dist-test-slave-fljv krb5kdc[20702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:41 dist-test-slave-fljv krb5kdc[20702](info): set up 2 sockets
Jun 19 18:47:41 dist-test-slave-fljv krb5kdc[20702](info): commencing operation
krb5kdc: starting...
W20260619 18:47:42.158465 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:42 dist-test-slave-fljv krb5kdc[20702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.254.keytab.
Entry for principal oryx/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.254.keytab.
I20260619 18:47:42.277287 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:44439
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.254.keytab
--principal=oryx/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:44439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:42.386927 20715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:42.387228 20715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:42.387267 20715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:42.391332 20715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:42.391405 20715 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 18:47:42.391422 20715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:42.391439 20715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:42.391454 20715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:42.391474 20715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:42.396992 20715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:44439
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.254.keytab
--principal=oryx/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:44439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:42.398113 20715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:42.399245 20715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:47:42.407949 20715 server_base.cc:1034] running on GCE node
W20260619 18:47:42.407910 20720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:42.407924 20723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:42.407897 20721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:42.408787 20715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:42.409948 20715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:42.411111 20715 hybrid_clock.cc:648] HybridClock initialized: now 1781894862411087 us; error 43 us; skew 500 ppm
Jun 19 18:47:42 dist-test-slave-fljv krb5kdc[20702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894862, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:42.419761 20715 init.cc:377] Logged in from keytab as oryx/127.12.234.254@KRBTEST.COM (short username oryx)
I20260619 18:47:42.420775 20715 webserver.cc:460] Webserver started at http://127.12.234.254:38281/ using document root <none> and password file <none>
I20260619 18:47:42.421404 20715 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:42.421476 20715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:42.421675 20715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:42.423846 20715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "1b076ecefbbf429d925e3e7da9b72351"
format_stamp: "Formatted at 2026-06-19 18:47:42 on dist-test-slave-fljv"
server_key: "2235bf17864cce19e62e344d49b92b15"
server_key_iv: "1f2554743deb9efdaf821536b80aadd7"
server_key_version: "encryptionkey@0"
I20260619 18:47:42.424401 20715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "1b076ecefbbf429d925e3e7da9b72351"
format_stamp: "Formatted at 2026-06-19 18:47:42 on dist-test-slave-fljv"
server_key: "2235bf17864cce19e62e344d49b92b15"
server_key_iv: "1f2554743deb9efdaf821536b80aadd7"
server_key_version: "encryptionkey@0"
I20260619 18:47:42.428290 20715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:42.430876 20730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:42.432222 20715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:42.432346 20715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "1b076ecefbbf429d925e3e7da9b72351"
format_stamp: "Formatted at 2026-06-19 18:47:42 on dist-test-slave-fljv"
server_key: "2235bf17864cce19e62e344d49b92b15"
server_key_iv: "1f2554743deb9efdaf821536b80aadd7"
server_key_version: "encryptionkey@0"
I20260619 18:47:42.432451 20715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:42.465162 20715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:42.468581 20715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:42.468789 20715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:42.476729 20715 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:44439
I20260619 18:47:42.476740 20792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:44439 every 8 connection(s)
I20260619 18:47:42.477711 20715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:42.480520 20793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:42.483995 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 20715
I20260619 18:47:42.484143 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:42.484450 13227 external_mini_cluster.cc:1428] Setting key 081f953dac66e433cc041e676393013f
I20260619 18:47:42.486752 20793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: Bootstrap starting.
I20260619 18:47:42.489005 20793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:42.489749 20793 log.cc:824] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:42.491777 20793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: No bootstrap required, opened a new log
Jun 19 18:47:42 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:42.495824 20793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b076ecefbbf429d925e3e7da9b72351" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44439 } }
I20260619 18:47:42.496129 20793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:42.496191 20793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b076ecefbbf429d925e3e7da9b72351, State: Initialized, Role: FOLLOWER
I20260619 18:47:42.496717 20793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b076ecefbbf429d925e3e7da9b72351" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44439 } }
I20260619 18:47:42.496876 20793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:42.496942 20793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:42.497040 20793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:42.498113 20793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b076ecefbbf429d925e3e7da9b72351" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44439 } }
I20260619 18:47:42.498458 20793 leader_election.cc:302] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b076ecefbbf429d925e3e7da9b72351; no voters:
I20260619 18:47:42.498795 20793 leader_election.cc:288] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:42.498904 20798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:42.499128 20798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [term 1 LEADER]: Becoming Leader. State: Replica: 1b076ecefbbf429d925e3e7da9b72351, State: Running, Role: LEADER
I20260619 18:47:42.499224 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:42.485945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.12.234.254:44439)
0619 18:47:42.486434 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:42.486450 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:42.486493 (+ 43us) server_negotiation.cc:407] Connection header received
0619 18:47:42.487201 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:42.487221 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:42.487588 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:42.487909 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:42.488717 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.489748 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.490671 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.491162 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.491312 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:42.493974 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:42.493979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:42.493994 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:42.494024 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:42.495874 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.496382 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.496388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.496393 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.496521 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.496802 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.496804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.496805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.497418 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:42.497573 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:42.497865 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 18:47:42.498229 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":178,"threads_started":1}
I20260619 18:47:42.499569 20798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b076ecefbbf429d925e3e7da9b72351" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44439 } }
I20260619 18:47:42.500123 20793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:42.501989 20800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b076ecefbbf429d925e3e7da9b72351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b076ecefbbf429d925e3e7da9b72351" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44439 } } }
I20260619 18:47:42.502118 20800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:42.502321 20799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b076ecefbbf429d925e3e7da9b72351. Latest consensus state: current_term: 1 leader_uuid: "1b076ecefbbf429d925e3e7da9b72351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b076ecefbbf429d925e3e7da9b72351" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 44439 } } }
I20260619 18:47:42.502486 20799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351 [sys.catalog]: This master's current role is: LEADER
W20260619 18:47:42.504958 20812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:47:42.505023 20812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:47:42.505105 20814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:42.505849 20814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:42.511752 20814 catalog_manager.cc:1347] Generated new cluster ID: 8dd02f71d3d44b5e93af366de84d479f
I20260619 18:47:42.511818 20814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:42.529835 20814 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:42.530908 20814 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:42.537322 20814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: Generated new TSK 0
I20260619 18:47:42.538070 20814 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab.
Entry for principal oryx/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab.
I20260619 18:47:42.569656 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44439
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab
--principal=oryx/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:42.683606 20819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:42.683884 20819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:42.683929 20819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:42.688024 20819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:42.688093 20819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:42.688133 20819 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 18:47:42.688164 20819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:42.693418 20819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab
--principal=oryx/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.254:44439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:42.694496 20819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:42.695657 20819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:42.706012 20825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:42.705956 20827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:42.705956 20824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:42.706251 20819 server_base.cc:1034] running on GCE node
I20260619 18:47:42.706772 20819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:42.707489 20819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:42.708658 20819 hybrid_clock.cc:648] HybridClock initialized: now 1781894862708637 us; error 36 us; skew 500 ppm
Jun 19 18:47:42 dist-test-slave-fljv krb5kdc[20702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894862, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:42.711483 20819 init.cc:377] Logged in from keytab as oryx/127.12.234.193@KRBTEST.COM (short username oryx)
I20260619 18:47:42.712435 20819 webserver.cc:460] Webserver started at http://127.12.234.193:37493/ using document root <none> and password file <none>
I20260619 18:47:42.712975 20819 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:42.713042 20819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:42.713232 20819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:42.715382 20819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "df41755a74c540d79683fecf6c4466ff"
format_stamp: "Formatted at 2026-06-19 18:47:42 on dist-test-slave-fljv"
server_key: "d018f93ae4f6bfc9777b8181ccb47342"
server_key_iv: "b3f32bcefc76e5d5b6aaba9717932a16"
server_key_version: "encryptionkey@0"
I20260619 18:47:42.715929 20819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "df41755a74c540d79683fecf6c4466ff"
format_stamp: "Formatted at 2026-06-19 18:47:42 on dist-test-slave-fljv"
server_key: "d018f93ae4f6bfc9777b8181ccb47342"
server_key_iv: "b3f32bcefc76e5d5b6aaba9717932a16"
server_key_version: "encryptionkey@0"
I20260619 18:47:42.719746 20819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:47:42.722314 20834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:42.723532 20819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:42.723632 20819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "df41755a74c540d79683fecf6c4466ff"
format_stamp: "Formatted at 2026-06-19 18:47:42 on dist-test-slave-fljv"
server_key: "d018f93ae4f6bfc9777b8181ccb47342"
server_key_iv: "b3f32bcefc76e5d5b6aaba9717932a16"
server_key_version: "encryptionkey@0"
I20260619 18:47:42.723745 20819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:42.737874 20819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:42.740988 20819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:42.741163 20819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:42.742650 20819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:42.742704 20819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:42.742758 20819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:42.742784 20819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 18:47:42 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894862, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:42.752941 20819 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:34835
I20260619 18:47:42.753039 20949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:34835 every 8 connection(s)
I20260619 18:47:42.753948 20819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:42.755949 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 20819
I20260619 18:47:42.756085 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:42.756399 13227 external_mini_cluster.cc:1428] Setting key fa32d310cedc95e35d51ababe69e5968
I20260619 18:47:42.757035 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:42.743339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:36735 (local address 127.12.234.254:44439)
0619 18:47:42.743475 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:42.743480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:42.744415 (+ 935us) server_negotiation.cc:407] Connection header received
0619 18:47:42.745284 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:42.745288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:42.745346 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:42.745452 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:42.747144 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.747940 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.749610 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.749969 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.750095 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:42.753288 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:42.753291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:42.753294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:42.753321 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:42.754936 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.755487 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.755490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.755491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.755573 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.755879 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.755882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.755884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.756184 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:42.756284 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:42.756777 (+ 493us) server_negotiation.cc:299] Negotiation successful
0619 18:47:42.756886 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 18:47:42.758036 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:42.743812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44439 (local address 127.12.234.193:36735)
0619 18:47:42.744275 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:42.744317 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:42.745081 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:42.745676 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:42.745686 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:42.746268 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:42.746944 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:42.746968 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.748081 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.748085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:42.749467 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:42.749480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.750149 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.750154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:42.750380 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:42.751041 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:42.751060 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:42.753107 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:42.755035 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:42.755039 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:42.755049 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:42.755398 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:42.755664 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:42.755666 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:42.755668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:42.755797 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:42.756277 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:42.756282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:42.756523 (+ 241us) client_negotiation.cc:769] Sending connection context
0619 18:47:42.756710 (+ 187us) client_negotiation.cc:240] Negotiation successful
0619 18:47:42.757027 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":105,"threads_started":1}
I20260619 18:47:42.759281 20950 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44439
I20260619 18:47:42.759433 20950 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:42.760040 20950 heartbeater.cc:511] Master 127.12.234.254:44439 requested a full tablet report, sending...
I20260619 18:47:42.761746 20746 ts_manager.cc:194] Registered new tserver with Master: df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835)
I20260619 18:47:42.763350 20746 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:36735
WARNING: no policy specified for oryx/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:47:42.768595 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:42.761538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:52787 (local address 127.12.234.254:44439)
0619 18:47:42.761663 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:42.761667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:42.761866 (+ 199us) server_negotiation.cc:407] Connection header received
0619 18:47:42.762025 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:42.762028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:42.762071 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:42.762134 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:42.762594 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.763223 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.764119 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.764350 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.764406 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:42.765625 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:42.765629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:42.765632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:42.765656 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:42.767050 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.767436 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.767439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.767441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.767520 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.767808 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.767812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.767814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.767988 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:42.768073 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:42.768317 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 18:47:42.768421 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":45}
I20260619 18:47:42.768680 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:42.761476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44439 (local address 127.12.234.193:52787)
0619 18:47:42.761796 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:42.761807 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:42.761932 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:42.762163 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:42.762166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:42.762387 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:42.762487 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:42.762493 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.763359 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.763362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:42.763985 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:42.763995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.764534 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.764536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:42.764606 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:42.765137 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:42.765150 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:42.765488 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:42.767151 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:42.767155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:42.767157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:42.767348 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:42.767610 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:42.767614 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:42.767616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:42.767717 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:42.768075 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:42.768079 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:42.768168 (+ 89us) client_negotiation.cc:769] Sending connection context
0619 18:47:42.768256 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 18:47:42.768446 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab.
Entry for principal oryx/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab.
I20260619 18:47:42.783222 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44439
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab
--principal=oryx/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:42.893075 20959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:42.893369 20959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:42.893414 20959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:42.897518 20959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:42.897595 20959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:42.897635 20959 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 18:47:42.897668 20959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:42.903268 20959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab
--principal=oryx/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.254:44439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:42.904356 20959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:42.905381 20959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:42.914997 20964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:42.915032 20965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:42.915014 20967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:42.915330 20959 server_base.cc:1034] running on GCE node
I20260619 18:47:42.915752 20959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:42.916424 20959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:42.917608 20959 hybrid_clock.cc:648] HybridClock initialized: now 1781894862917574 us; error 33 us; skew 500 ppm
Jun 19 18:47:42 dist-test-slave-fljv krb5kdc[20702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894862, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:42.920382 20959 init.cc:377] Logged in from keytab as oryx/127.12.234.194@KRBTEST.COM (short username oryx)
I20260619 18:47:42.921278 20959 webserver.cc:460] Webserver started at http://127.12.234.194:46325/ using document root <none> and password file <none>
I20260619 18:47:42.921844 20959 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:42.921904 20959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:42.922087 20959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:42.924137 20959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "b32c71155665486dbadacafd9618f3ce"
format_stamp: "Formatted at 2026-06-19 18:47:42 on dist-test-slave-fljv"
server_key: "c28678094ec2741fbc1d36666089e4ef"
server_key_iv: "04d96d9c1360b2d5ffbd3941c6fda627"
server_key_version: "encryptionkey@0"
I20260619 18:47:42.924659 20959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "b32c71155665486dbadacafd9618f3ce"
format_stamp: "Formatted at 2026-06-19 18:47:42 on dist-test-slave-fljv"
server_key: "c28678094ec2741fbc1d36666089e4ef"
server_key_iv: "04d96d9c1360b2d5ffbd3941c6fda627"
server_key_version: "encryptionkey@0"
I20260619 18:47:42.928349 20959 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 18:47:42.930800 20974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:42.932138 20959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:42.932241 20959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "b32c71155665486dbadacafd9618f3ce"
format_stamp: "Formatted at 2026-06-19 18:47:42 on dist-test-slave-fljv"
server_key: "c28678094ec2741fbc1d36666089e4ef"
server_key_iv: "04d96d9c1360b2d5ffbd3941c6fda627"
server_key_version: "encryptionkey@0"
I20260619 18:47:42.932346 20959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:42.954101 20959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:42.957206 20959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:42.957382 20959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:42.958931 20959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:42.958992 20959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:42.959056 20959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:42.959084 20959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 18:47:42 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894862, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:42.968469 20959 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:39933
I20260619 18:47:42.968505 21089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:39933 every 8 connection(s)
I20260619 18:47:42.969489 20959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:42.972870 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:42.959683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:59467 (local address 127.12.234.254:44439)
0619 18:47:42.959820 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:42.959824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:42.960756 (+ 932us) server_negotiation.cc:407] Connection header received
0619 18:47:42.961820 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:42.961824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:42.961879 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:42.961940 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:42.963630 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.964276 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.965928 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.966160 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.966276 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:42.969209 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:42.969212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:42.969214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:42.969241 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:42.970909 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.971402 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.971405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.971407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.971498 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.971840 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.971843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.971844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.972092 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:42.972179 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:42.972636 (+ 457us) server_negotiation.cc:299] Negotiation successful
0619 18:47:42.972743 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 18:47:42.973713 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:42.960148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44439 (local address 127.12.234.194:59467)
0619 18:47:42.960600 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:42.960649 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:42.961563 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:42.962381 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:42.962391 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:42.962805 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:42.963483 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:42.963505 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.964396 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.964399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:42.965814 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:42.965825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.966327 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.966330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:42.966529 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:42.967223 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:42.967244 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:42.968987 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:42.971029 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:42.971033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:42.971045 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:42.971309 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:42.971588 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:42.971590 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:42.971591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:42.971745 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:42.972183 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:42.972189 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:42.972417 (+ 228us) client_negotiation.cc:769] Sending connection context
0619 18:47:42.972567 (+ 150us) client_negotiation.cc:240] Negotiation successful
0619 18:47:42.972847 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
I20260619 18:47:42.975032 21090 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44439
I20260619 18:47:42.975170 21090 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:42.975653 21090 heartbeater.cc:511] Master 127.12.234.254:44439 requested a full tablet report, sending...
I20260619 18:47:42.976604 20747 ts_manager.cc:194] Registered new tserver with Master: b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933)
I20260619 18:47:42.977272 20747 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:59467
I20260619 18:47:42.979447 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 20959
I20260619 18:47:42.979584 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:42.979861 13227 external_mini_cluster.cc:1428] Setting key e8ac522364e85e3596371c4c4aa3cec5
I20260619 18:47:42.983798 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:42.976990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:54227 (local address 127.12.234.254:44439)
0619 18:47:42.977107 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:42.977111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:42.977307 (+ 196us) server_negotiation.cc:407] Connection header received
0619 18:47:42.977435 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:42.977438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:42.977478 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:42.977540 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:42.978173 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.978698 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.979560 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.979819 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.979904 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:42.981008 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:42.981012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:42.981015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:42.981037 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:42.982337 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.982710 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.982713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.982715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.982785 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:42.983017 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:42.983020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:42.983022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:42.983226 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:42.983311 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:42.983549 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 18:47:42.983658 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 18:47:42.983810 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:42.976911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44439 (local address 127.12.234.194:54227)
0619 18:47:42.977233 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:42.977244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:42.977329 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:42.977787 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:42.977789 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:42.977946 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:42.978072 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:42.978077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.978808 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.978811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:42.979439 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:42.979450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:42.979926 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:42.979929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:42.980019 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:42.980568 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:42.980578 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:42.980882 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:42.982433 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:42.982436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:42.982438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:42.982624 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:42.982862 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:42.982865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:42.982867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:42.982944 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:42.983316 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:42.983319 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:42.983406 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 18:47:42.983487 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 18:47:42.983625 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for oryx/127.12.234.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab.
Entry for principal oryx/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab.
I20260619 18:47:43.006067 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:44439
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab
--principal=oryx/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:43.117694 21099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:43.117977 21099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:43.118022 21099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:43.122082 21099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:43.122145 21099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:43.122215 21099 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 18:47:43.122265 21099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:43.127650 21099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab
--principal=oryx/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.254:44439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:43.128750 21099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:43.129781 21099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:43.139931 21104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:43.140041 21105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:43.140129 21099 server_base.cc:1034] running on GCE node
W20260619 18:47:43.140223 21107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:43.140764 21099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:43.141521 21099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:43.142691 21099 hybrid_clock.cc:648] HybridClock initialized: now 1781894863142664 us; error 45 us; skew 500 ppm
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894863, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:43.145638 21099 init.cc:377] Logged in from keytab as oryx/127.12.234.195@KRBTEST.COM (short username oryx)
I20260619 18:47:43.146668 21099 webserver.cc:460] Webserver started at http://127.12.234.195:37233/ using document root <none> and password file <none>
I20260619 18:47:43.147403 21099 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:43.147486 21099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:43.147723 21099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:43.150367 21099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "9badb6e4da614640b2debf2778ba6521"
format_stamp: "Formatted at 2026-06-19 18:47:43 on dist-test-slave-fljv"
server_key: "7fdede142f7a5aac84be7112f2e5cc81"
server_key_iv: "c9b42e70c1d6a84a85e149f0e5e176e1"
server_key_version: "encryptionkey@0"
I20260619 18:47:43.150950 21099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "9badb6e4da614640b2debf2778ba6521"
format_stamp: "Formatted at 2026-06-19 18:47:43 on dist-test-slave-fljv"
server_key: "7fdede142f7a5aac84be7112f2e5cc81"
server_key_iv: "c9b42e70c1d6a84a85e149f0e5e176e1"
server_key_version: "encryptionkey@0"
I20260619 18:47:43.154834 21099 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:47:43.157425 21114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:43.158744 21099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:43.158843 21099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "9badb6e4da614640b2debf2778ba6521"
format_stamp: "Formatted at 2026-06-19 18:47:43 on dist-test-slave-fljv"
server_key: "7fdede142f7a5aac84be7112f2e5cc81"
server_key_iv: "c9b42e70c1d6a84a85e149f0e5e176e1"
server_key_version: "encryptionkey@0"
I20260619 18:47:43.158934 21099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:43.171118 21099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:43.173802 21099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:43.173947 21099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:43.175356 21099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:43.175406 21099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:43.175457 21099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:43.175482 21099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894863, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:43.185300 21099 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:37785
I20260619 18:47:43.185353 21229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:37785 every 8 connection(s)
I20260619 18:47:43.186159 21099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:43.187779 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.176147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:59967 (local address 127.12.234.254:44439)
0619 18:47:43.176272 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.176276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.177365 (+ 1089us) server_negotiation.cc:407] Connection header received
0619 18:47:43.178421 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.178426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.178488 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.178597 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:43.179902 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.180427 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.181523 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.181791 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.181878 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.184293 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:43.184297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:43.184300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:43.184325 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:43.185772 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.186256 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.186260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.186263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.186341 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.186700 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.186717 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.186719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.186940 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:43.187030 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.187502 (+ 472us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.187618 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 18:47:43.188521 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.176737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44439 (local address 127.12.234.195:59967)
0619 18:47:43.177219 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.177261 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.178153 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.178794 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.178804 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.179205 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:43.179762 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.179777 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.180541 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.180544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.181403 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.181412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.181965 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.181970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.182139 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.182694 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:43.182712 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:43.184123 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:43.185872 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:43.185877 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:43.185891 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:43.186150 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:43.186421 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:43.186424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:43.186426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:43.186591 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:43.187035 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:43.187041 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:43.187271 (+ 230us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.187426 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.187737 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":122,"threads_started":1}
I20260619 18:47:43.189594 21230 heartbeater.cc:347] Connected to a master server at 127.12.234.254:44439
I20260619 18:47:43.189855 21230 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:43.190361 21230 heartbeater.cc:511] Master 127.12.234.254:44439 requested a full tablet report, sending...
I20260619 18:47:43.191344 20746 ts_manager.cc:194] Registered new tserver with Master: 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785)
I20260619 18:47:43.191984 20746 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:59967
I20260619 18:47:43.192343 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 21099
I20260619 18:47:43.192467 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:43.192766 13227 external_mini_cluster.cc:1428] Setting key 55f4f43e05507086ae945b38d8cfe6ab
I20260619 18:47:43.194787 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:47:43.199472 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.191909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:37919 (local address 127.12.234.254:44439)
0619 18:47:43.192018 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.192022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.192254 (+ 232us) server_negotiation.cc:407] Connection header received
0619 18:47:43.192374 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.192377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.192419 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.192503 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:43.192954 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.193623 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.195266 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.195554 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.195651 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.196665 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:43.196669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:43.196672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:43.196697 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:43.197943 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.198399 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.198402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.198403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.198472 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.198743 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.198746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.198748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.198905 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:43.198987 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.199227 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.199322 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 18:47:43.199493 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.191834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44439 (local address 127.12.234.195:37919)
0619 18:47:43.192184 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.192197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.192290 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.192517 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.192520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.192731 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:43.192840 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.192847 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.193748 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.193752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.195145 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.195154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.195662 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.195665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.195739 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.196235 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:43.196246 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:43.196537 (+ 291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:43.198044 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:43.198047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:43.198050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:43.198320 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:43.198554 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:43.198557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:43.198560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:43.198661 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:43.198988 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:43.198991 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:43.199077 (+ 86us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.199167 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.199306 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":73,"threads_started":1}
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894863, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 18:47:43.218812 20796 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:43.212106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.12.234.254:44439)
0619 18:47:43.212240 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.212243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.212255 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:47:43.212291 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.212293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.212330 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.212396 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:43.213067 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.213580 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.214466 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.214677 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.214765 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.218654 (+ 3889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37298: BlockingRecv error: recv got EOF from 127.0.0.1:37298 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894863, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:43.229568 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.221096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.12.234.254:44439)
0619 18:47:43.221229 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.221234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.221673 (+ 439us) server_negotiation.cc:407] Connection header received
0619 18:47:43.221816 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.221821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.221869 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.221915 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:43.222712 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.223383 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.224212 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.224420 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.224498 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.226689 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:43.226692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:43.226694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:43.226729 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:43.228146 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.228524 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.228528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.228530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.228604 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.228826 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.228828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.228829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.229005 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:43.229108 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.229321 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.229415 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 18:47:43.232499 20746 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:43.235636 20746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:43.248957 21252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.243442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34835 (local address 127.0.0.1:47266)
0619 18:47:43.243855 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.243905 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.244031 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.244762 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.244767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.244788 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:43.244915 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.244923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.246108 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.246111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.247216 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.247226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.248442 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.248446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.248551 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.248587 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.248696 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.248806 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":43,"thread_start_us":62,"threads_started":1}
I20260619 18:47:43.248988 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.243602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.12.234.193:34835)
0619 18:47:43.243962 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.243978 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.244000 (+ 22us) server_negotiation.cc:407] Connection header received
0619 18:47:43.244225 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.244231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.244443 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.244587 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:43.245037 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.246021 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.247347 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.248120 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.248249 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.248383 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.248747 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.248846 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":88,"threads_started":1}
I20260619 18:47:43.249519 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.244117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37785 (local address 127.0.0.1:39320)
0619 18:47:43.244872 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.244883 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.244964 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.245396 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.245399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.245414 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:43.245501 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.245508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.246917 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.246920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.248054 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.248065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.249174 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.249178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.249245 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.249262 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.249328 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.249393 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":4,"thread_start_us":75,"threads_started":1}
I20260619 18:47:43.249588 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.244287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.12.234.195:37785)
0619 18:47:43.244679 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.244689 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.244958 (+ 269us) server_negotiation.cc:407] Connection header received
0619 18:47:43.245061 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.245066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.245259 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.245410 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:43.245618 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.246806 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.248189 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.248745 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.248867 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.248977 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.249374 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.249467 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":113,"threads_started":1}
I20260619 18:47:43.249848 21253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.243511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:39933 (local address 127.0.0.1:47594)
0619 18:47:43.244632 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.244646 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.244721 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.245566 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.245569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.245585 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:43.245662 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.245667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.246979 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.246983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.248103 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.248112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.249307 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.249311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.249404 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.249426 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.249481 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.249543 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"mutex_wait_us":20,"thread_start_us":73,"threads_started":1}
I20260619 18:47:43.250471 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.244202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47594 (local address 127.12.234.194:39933)
0619 18:47:43.245181 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.245189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.245206 (+ 17us) server_negotiation.cc:407] Connection header received
0619 18:47:43.245260 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.245263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.245413 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.245598 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:43.245754 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.246882 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.248215 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.248981 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.250082 (+ 1101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.250205 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.250254 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.250338 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":91,"threads_started":1}
I20260619 18:47:43.252167 21164 tablet_service.cc:1467] Processing CreateTablet for tablet e86661f0959646d0982afd3b6fdbf967 (DEFAULT_TABLE table=test-table [id=339c80f7e67a45a485fd477c2371ddd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:43.252171 20884 tablet_service.cc:1467] Processing CreateTablet for tablet e86661f0959646d0982afd3b6fdbf967 (DEFAULT_TABLE table=test-table [id=339c80f7e67a45a485fd477c2371ddd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:43.252710 21023 tablet_service.cc:1467] Processing CreateTablet for tablet e86661f0959646d0982afd3b6fdbf967 (DEFAULT_TABLE table=test-table [id=339c80f7e67a45a485fd477c2371ddd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:43.253255 20884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e86661f0959646d0982afd3b6fdbf967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:43.253268 21164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e86661f0959646d0982afd3b6fdbf967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:43.253548 21023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e86661f0959646d0982afd3b6fdbf967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:43.260380 21258 tablet_bootstrap.cc:492] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: Bootstrap starting.
I20260619 18:47:43.261853 21259 tablet_bootstrap.cc:492] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: Bootstrap starting.
I20260619 18:47:43.262555 21260 tablet_bootstrap.cc:492] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: Bootstrap starting.
I20260619 18:47:43.262930 21258 tablet_bootstrap.cc:654] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:43.263758 21259 tablet_bootstrap.cc:654] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:43.263836 21258 log.cc:824] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:43.264497 21259 log.cc:824] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:43.264565 21260 tablet_bootstrap.cc:654] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:43.265329 21260 log.cc:824] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:43.266271 21259 tablet_bootstrap.cc:492] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: No bootstrap required, opened a new log
I20260619 18:47:43.266273 21258 tablet_bootstrap.cc:492] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: No bootstrap required, opened a new log
I20260619 18:47:43.266458 21259 ts_tablet_manager.cc:1397] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 18:47:43.266460 21258 ts_tablet_manager.cc:1397] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:47:43.267717 21260 tablet_bootstrap.cc:492] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: No bootstrap required, opened a new log
I20260619 18:47:43.267885 21260 ts_tablet_manager.cc:1397] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260619 18:47:43.269398 21259 raft_consensus.cc:348] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.269624 21259 raft_consensus.cc:374] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:43.269676 21259 raft_consensus.cc:729] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df41755a74c540d79683fecf6c4466ff, State: Initialized, Role: FOLLOWER
I20260619 18:47:43.270074 21259 consensus_queue.cc:260] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.270215 21258 raft_consensus.cc:348] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.270469 21258 raft_consensus.cc:374] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:43.270524 21258 raft_consensus.cc:729] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32c71155665486dbadacafd9618f3ce, State: Initialized, Role: FOLLOWER
I20260619 18:47:43.270784 20950 heartbeater.cc:503] Master 127.12.234.254:44439 was elected leader, sending a full tablet report...
I20260619 18:47:43.270939 21258 consensus_queue.cc:260] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.271016 21259 ts_tablet_manager.cc:1428] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260619 18:47:43.271507 21090 heartbeater.cc:503] Master 127.12.234.254:44439 was elected leader, sending a full tablet report...
I20260619 18:47:43.272436 21258 ts_tablet_manager.cc:1428] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:47:43.273451 21260 raft_consensus.cc:348] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.273653 21260 raft_consensus.cc:374] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:43.273705 21260 raft_consensus.cc:729] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9badb6e4da614640b2debf2778ba6521, State: Initialized, Role: FOLLOWER
I20260619 18:47:43.274121 21260 consensus_queue.cc:260] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.275068 21260 ts_tablet_manager.cc:1428] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260619 18:47:43.275174 21230 heartbeater.cc:503] Master 127.12.234.254:44439 was elected leader, sending a full tablet report...
W20260619 18:47:43.437132 21231 tablet.cc:2369] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:47:43.470891 21091 tablet.cc:2369] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:47:43.505313 20951 tablet.cc:2369] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:47:43.538625 21265 raft_consensus.cc:482] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:43.538846 21265 raft_consensus.cc:504] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.539969 21265 leader_election.cc:288] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835), 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785)
I20260619 18:47:43.545238 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.540769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:46619 (local address 127.12.234.195:37785)
0619 18:47:43.540909 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.540913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.541070 (+ 157us) server_negotiation.cc:407] Connection header received
0619 18:47:43.541123 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.541125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.541160 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.541228 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:43.541420 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.542138 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.543884 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.544535 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.544684 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.544745 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.545019 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.545094 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 18:47:43.545250 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.540415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:60409 (local address 127.12.234.193:34835)
0619 18:47:43.540576 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.540580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.540672 (+ 92us) server_negotiation.cc:407] Connection header received
0619 18:47:43.540823 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.540827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.540874 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.540948 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:43.541247 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.542020 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.543884 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.544535 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.544684 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.544745 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.545019 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.545104 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 18:47:43.545270 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.540289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34835 (local address 127.12.234.194:60409)
0619 18:47:43.540575 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.540590 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.540718 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.540982 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.540985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.541018 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:43.541142 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.541147 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.542219 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.542223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.543717 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.543730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.544701 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.544706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.544878 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.544911 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.544982 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.545118 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":153,"thread_start_us":92,"threads_started":1}
I20260619 18:47:43.545269 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.540647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37785 (local address 127.12.234.194:46619)
0619 18:47:43.540929 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.540943 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.541029 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.541248 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.541250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.541266 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:43.541336 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.541340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.542263 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.542266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.543718 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.543730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.544702 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.544706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.544873 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.544904 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.544982 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.545078 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":85,"thread_start_us":65,"threads_started":1}
I20260619 18:47:43.545935 20904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86661f0959646d0982afd3b6fdbf967" candidate_uuid: "b32c71155665486dbadacafd9618f3ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df41755a74c540d79683fecf6c4466ff" is_pre_election: true
I20260619 18:47:43.546307 20904 raft_consensus.cc:2454] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32c71155665486dbadacafd9618f3ce in term 0.
I20260619 18:47:43.546356 21184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86661f0959646d0982afd3b6fdbf967" candidate_uuid: "b32c71155665486dbadacafd9618f3ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9badb6e4da614640b2debf2778ba6521" is_pre_election: true
I20260619 18:47:43.546635 21184 raft_consensus.cc:2454] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32c71155665486dbadacafd9618f3ce in term 0.
I20260619 18:47:43.546821 20975 leader_election.cc:302] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32c71155665486dbadacafd9618f3ce, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:43.547132 21265 raft_consensus.cc:2781] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:43.547209 21265 raft_consensus.cc:482] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:43.547237 21265 raft_consensus.cc:3037] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:43.548275 21265 raft_consensus.cc:504] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.548602 21265 leader_election.cc:288] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [CANDIDATE]: Term 1 election: Requested vote from peers df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835), 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785)
I20260619 18:47:43.548964 20904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86661f0959646d0982afd3b6fdbf967" candidate_uuid: "b32c71155665486dbadacafd9618f3ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df41755a74c540d79683fecf6c4466ff"
I20260619 18:47:43.548981 21184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86661f0959646d0982afd3b6fdbf967" candidate_uuid: "b32c71155665486dbadacafd9618f3ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9badb6e4da614640b2debf2778ba6521"
I20260619 18:47:43.549113 21184 raft_consensus.cc:3037] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:43.549100 20904 raft_consensus.cc:3037] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:43.550457 20904 raft_consensus.cc:2454] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32c71155665486dbadacafd9618f3ce in term 1.
I20260619 18:47:43.550705 21184 raft_consensus.cc:2454] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32c71155665486dbadacafd9618f3ce in term 1.
I20260619 18:47:43.550778 20975 leader_election.cc:302] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32c71155665486dbadacafd9618f3ce, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:43.550946 21265 raft_consensus.cc:2781] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:43.551151 21265 raft_consensus.cc:686] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 1 LEADER]: Becoming Leader. State: Replica: b32c71155665486dbadacafd9618f3ce, State: Running, Role: LEADER
I20260619 18:47:43.551476 21265 consensus_queue.cc:237] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.554811 20746 catalog_manager.cc:5495] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce reported cstate change: term changed from 0 to 1, leader changed from <none> to b32c71155665486dbadacafd9618f3ce (127.12.234.194). New cstate: current_term: 1 leader_uuid: "b32c71155665486dbadacafd9618f3ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260619 18:47:43.612524 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.609040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.12.234.194:39933)
0619 18:47:43.609205 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.609209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.609224 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:43.609356 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.609359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.609413 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.609505 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:43.609764 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.610303 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.611128 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.611343 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.611429 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.611696 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:43.612149 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:43.612250 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.612323 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.612383 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 18:47:43.618779 21184 raft_consensus.cc:1261] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Refusing update from remote peer b32c71155665486dbadacafd9618f3ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:43.618779 20904 raft_consensus.cc:1261] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Refusing update from remote peer b32c71155665486dbadacafd9618f3ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:43.619590 21269 consensus_queue.cc:1035] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:43.619783 21265 consensus_queue.cc:1035] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:43.639911 20746 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:43.640096 20746 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:43.642743 20746 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: Sending DeleteTablet for 3 replicas of tablet e86661f0959646d0982afd3b6fdbf967
I20260619 18:47:43.643702 21023 tablet_service.cc:1514] Processing DeleteTablet for tablet e86661f0959646d0982afd3b6fdbf967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:43 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47594
I20260619 18:47:43.643755 21164 tablet_service.cc:1514] Processing DeleteTablet for tablet e86661f0959646d0982afd3b6fdbf967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:43 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:39320
I20260619 18:47:43.643805 20884 tablet_service.cc:1514] Processing DeleteTablet for tablet e86661f0959646d0982afd3b6fdbf967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:43 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47266
I20260619 18:47:43.644289 21284 tablet_replica.cc:330] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: stopping tablet replica
I20260619 18:47:43.644660 21283 tablet_replica.cc:330] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: stopping tablet replica
I20260619 18:47:43.644652 21284 raft_consensus.cc:2229] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:43.645032 21283 raft_consensus.cc:2229] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:43.645200 21284 raft_consensus.cc:2258] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:43.645143 20746 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 18:47:43.645615 21282 tablet_replica.cc:330] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: stopping tablet replica
W20260619 18:47:43.645630 20746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:43.645864 21283 raft_consensus.cc:2258] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:43.645920 21282 raft_consensus.cc:2229] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:43.646258 21282 raft_consensus.cc:2258] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:43.647662 21283 ts_tablet_manager.cc:1905] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:43.648216 21282 ts_tablet_manager.cc:1905] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:43.648905 21284 ts_tablet_manager.cc:1905] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:43.651015 21282 ts_tablet_manager.cc:1918] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:43.651134 21282 log.cc:1196] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/e86661f0959646d0982afd3b6fdbf967
I20260619 18:47:43.651122 21023 tablet_service.cc:1467] Processing CreateTablet for tablet 10418ed1508a4c4db9281ec6dd806db5 (DEFAULT_TABLE table=test-table [id=3001911f5f1f4772b5ba529ebecfa754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:43.651422 21282 ts_tablet_manager.cc:1939] T e86661f0959646d0982afd3b6fdbf967 P 9badb6e4da614640b2debf2778ba6521: Deleting consensus metadata
I20260619 18:47:43.651921 20884 tablet_service.cc:1467] Processing CreateTablet for tablet 10418ed1508a4c4db9281ec6dd806db5 (DEFAULT_TABLE table=test-table [id=3001911f5f1f4772b5ba529ebecfa754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:43.652168 20884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10418ed1508a4c4db9281ec6dd806db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:43.653751 21284 ts_tablet_manager.cc:1918] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:43.653843 21284 log.cc:1196] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/e86661f0959646d0982afd3b6fdbf967
I20260619 18:47:43.654093 21284 ts_tablet_manager.cc:1939] T e86661f0959646d0982afd3b6fdbf967 P b32c71155665486dbadacafd9618f3ce: Deleting consensus metadata
I20260619 18:47:43.654400 21259 tablet_bootstrap.cc:492] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: Bootstrap starting.
I20260619 18:47:43.654783 20731 catalog_manager.cc:4841] TS 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785): tablet e86661f0959646d0982afd3b6fdbf967 (table test-table [id=339c80f7e67a45a485fd477c2371ddd7]) successfully deleted
I20260619 18:47:43.655103 21023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10418ed1508a4c4db9281ec6dd806db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:43.655328 20733 catalog_manager.cc:4841] TS b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933): tablet e86661f0959646d0982afd3b6fdbf967 (table test-table [id=339c80f7e67a45a485fd477c2371ddd7]) successfully deleted
I20260619 18:47:43.655403 21259 tablet_bootstrap.cc:654] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:43.656114 21283 ts_tablet_manager.cc:1918] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:43.656203 21283 log.cc:1196] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/e86661f0959646d0982afd3b6fdbf967
I20260619 18:47:43.656477 21283 ts_tablet_manager.cc:1939] T e86661f0959646d0982afd3b6fdbf967 P df41755a74c540d79683fecf6c4466ff: Deleting consensus metadata
I20260619 18:47:43.657249 21258 tablet_bootstrap.cc:492] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: Bootstrap starting.
I20260619 18:47:43.657476 21164 tablet_service.cc:1467] Processing CreateTablet for tablet 10418ed1508a4c4db9281ec6dd806db5 (DEFAULT_TABLE table=test-table [id=3001911f5f1f4772b5ba529ebecfa754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:43.657727 21164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10418ed1508a4c4db9281ec6dd806db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:43.658209 21258 tablet_bootstrap.cc:654] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:43.658210 20731 catalog_manager.cc:4841] TS df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835): tablet e86661f0959646d0982afd3b6fdbf967 (table test-table [id=339c80f7e67a45a485fd477c2371ddd7]) successfully deleted
I20260619 18:47:43.658972 21259 tablet_bootstrap.cc:492] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: No bootstrap required, opened a new log
I20260619 18:47:43.659067 21259 ts_tablet_manager.cc:1397] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 18:47:43.659291 21258 tablet_bootstrap.cc:492] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: No bootstrap required, opened a new log
I20260619 18:47:43.659358 21258 ts_tablet_manager.cc:1397] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 18:47:43.659498 21259 raft_consensus.cc:348] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.659601 21259 raft_consensus.cc:374] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:43.659631 21259 raft_consensus.cc:729] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df41755a74c540d79683fecf6c4466ff, State: Initialized, Role: FOLLOWER
I20260619 18:47:43.659767 21259 consensus_queue.cc:260] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.659794 21258 raft_consensus.cc:348] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.659878 21258 raft_consensus.cc:374] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:43.659907 21258 raft_consensus.cc:729] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32c71155665486dbadacafd9618f3ce, State: Initialized, Role: FOLLOWER
I20260619 18:47:43.659945 21260 tablet_bootstrap.cc:492] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: Bootstrap starting.
I20260619 18:47:43.660038 21259 ts_tablet_manager.cc:1428] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:43.660010 21258 consensus_queue.cc:260] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.660269 21258 ts_tablet_manager.cc:1428] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 18:47:43.660880 21260 tablet_bootstrap.cc:654] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:43.662257 21260 tablet_bootstrap.cc:492] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: No bootstrap required, opened a new log
I20260619 18:47:43.662343 21260 ts_tablet_manager.cc:1397] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260619 18:47:43.662779 21260 raft_consensus.cc:348] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.662892 21260 raft_consensus.cc:374] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:43.662952 21260 raft_consensus.cc:729] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9badb6e4da614640b2debf2778ba6521, State: Initialized, Role: FOLLOWER
I20260619 18:47:43.663065 21260 consensus_queue.cc:260] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.663319 21260 ts_tablet_manager.cc:1428] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 18:47:43.717027 21265 raft_consensus.cc:482] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:43.717181 21265 raft_consensus.cc:504] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.717561 21265 leader_election.cc:288] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835), 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785)
I20260619 18:47:43.718030 21184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10418ed1508a4c4db9281ec6dd806db5" candidate_uuid: "b32c71155665486dbadacafd9618f3ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9badb6e4da614640b2debf2778ba6521" is_pre_election: true
I20260619 18:47:43.718024 20904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10418ed1508a4c4db9281ec6dd806db5" candidate_uuid: "b32c71155665486dbadacafd9618f3ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df41755a74c540d79683fecf6c4466ff" is_pre_election: true
I20260619 18:47:43.718197 21184 raft_consensus.cc:2454] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32c71155665486dbadacafd9618f3ce in term 0.
I20260619 18:47:43.718210 20904 raft_consensus.cc:2454] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32c71155665486dbadacafd9618f3ce in term 0.
I20260619 18:47:43.718506 20975 leader_election.cc:302] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32c71155665486dbadacafd9618f3ce, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:43.718701 21265 raft_consensus.cc:2781] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:43.718760 21265 raft_consensus.cc:482] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:43.718804 21265 raft_consensus.cc:3037] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:43.719811 21265 raft_consensus.cc:504] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.720116 21265 leader_election.cc:288] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [CANDIDATE]: Term 1 election: Requested vote from peers df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835), 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785)
I20260619 18:47:43.720548 20904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10418ed1508a4c4db9281ec6dd806db5" candidate_uuid: "b32c71155665486dbadacafd9618f3ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df41755a74c540d79683fecf6c4466ff"
I20260619 18:47:43.720518 21184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10418ed1508a4c4db9281ec6dd806db5" candidate_uuid: "b32c71155665486dbadacafd9618f3ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9badb6e4da614640b2debf2778ba6521"
I20260619 18:47:43.720660 20904 raft_consensus.cc:3037] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:43.720660 21184 raft_consensus.cc:3037] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:43.721748 20904 raft_consensus.cc:2454] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32c71155665486dbadacafd9618f3ce in term 1.
I20260619 18:47:43.721750 21184 raft_consensus.cc:2454] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32c71155665486dbadacafd9618f3ce in term 1.
I20260619 18:47:43.722055 20975 leader_election.cc:302] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32c71155665486dbadacafd9618f3ce, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:43.722229 21265 raft_consensus.cc:2781] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:43.722329 21265 raft_consensus.cc:686] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 1 LEADER]: Becoming Leader. State: Replica: b32c71155665486dbadacafd9618f3ce, State: Running, Role: LEADER
I20260619 18:47:43.722467 21265 consensus_queue.cc:237] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.724123 20746 catalog_manager.cc:5495] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce reported cstate change: term changed from 0 to 1, leader changed from <none> to b32c71155665486dbadacafd9618f3ce (127.12.234.194). New cstate: current_term: 1 leader_uuid: "b32c71155665486dbadacafd9618f3ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } health_report { overall_health: UNKNOWN } } }
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894862, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.254@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:43.757948 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.747910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:44439 (local address 127.0.0.1:37322)
0619 18:47:43.748254 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.748270 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.748403 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.748667 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.748671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.749209 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:43.749377 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.749406 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.750224 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.750228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.751092 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.751104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.751670 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.751673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.751726 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.752255 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:43.752268 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:43.753975 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:43.756010 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:43.756019 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:43.756025 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:43.756309 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:43.756638 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:43.756641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:43.756644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:43.756740 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:43.757264 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:43.757270 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:43.757393 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.757518 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.757710 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":115,"threads_started":1}
I20260619 18:47:43.757975 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.747989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37322 (local address 127.12.234.254:44439)
0619 18:47:43.748297 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.748304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.748339 (+ 35us) server_negotiation.cc:407] Connection header received
0619 18:47:43.748492 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.748494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.748533 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.748609 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:43.749536 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.750076 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.751209 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.751479 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.751569 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.754123 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:43.754126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:43.754129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:43.754163 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:43.755898 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.756416 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.756420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.756422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.756526 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.756837 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.756851 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.756853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.757112 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:43.757215 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.757582 (+ 367us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.757786 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":248,"thread_start_us":71,"threads_started":1}
I20260619 18:47:43.766887 20746 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:37322:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 18:47:43.767765 20746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:43.772588 20884 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb4c344aeb14aa888e16249cc530eb4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e3ebf04aa3f447f0ab7dee67ecca4c2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 18:47:43.772702 21023 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb4c344aeb14aa888e16249cc530eb4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e3ebf04aa3f447f0ab7dee67ecca4c2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 18:47:43.772889 21164 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb4c344aeb14aa888e16249cc530eb4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e3ebf04aa3f447f0ab7dee67ecca4c2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 18:47:43.773037 20884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb4c344aeb14aa888e16249cc530eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:43.773116 21023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb4c344aeb14aa888e16249cc530eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:43.773218 21164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb4c344aeb14aa888e16249cc530eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:43.776109 21258 tablet_bootstrap.cc:492] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: Bootstrap starting.
I20260619 18:47:43.776196 21259 tablet_bootstrap.cc:492] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: Bootstrap starting.
I20260619 18:47:43.777048 21258 tablet_bootstrap.cc:654] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:43.777046 21259 tablet_bootstrap.cc:654] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:43.778568 21260 tablet_bootstrap.cc:492] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: Bootstrap starting.
I20260619 18:47:43.778800 21258 tablet_bootstrap.cc:492] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: No bootstrap required, opened a new log
I20260619 18:47:43.778887 21258 ts_tablet_manager.cc:1397] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260619 18:47:43.779318 21258 raft_consensus.cc:348] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.779436 21258 raft_consensus.cc:374] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:43.779451 21260 tablet_bootstrap.cc:654] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:43.779464 21258 raft_consensus.cc:729] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32c71155665486dbadacafd9618f3ce, State: Initialized, Role: FOLLOWER
I20260619 18:47:43.779742 21258 consensus_queue.cc:260] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.780025 21258 ts_tablet_manager.cc:1428] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:43.780020 21265 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.780206 21265 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:43.781347 21260 tablet_bootstrap.cc:492] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: No bootstrap required, opened a new log
I20260619 18:47:43.781428 21260 ts_tablet_manager.cc:1397] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:43.781582 21259 tablet_bootstrap.cc:492] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: No bootstrap required, opened a new log
I20260619 18:47:43.781668 21259 ts_tablet_manager.cc:1397] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 18:47:43.781867 21260 raft_consensus.cc:348] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.781984 21260 raft_consensus.cc:374] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:43.782011 21260 raft_consensus.cc:729] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9badb6e4da614640b2debf2778ba6521, State: Initialized, Role: FOLLOWER
I20260619 18:47:43.782150 21260 consensus_queue.cc:260] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.782111 21259 raft_consensus.cc:348] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.782310 21259 raft_consensus.cc:374] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:43.782347 21259 raft_consensus.cc:729] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df41755a74c540d79683fecf6c4466ff, State: Initialized, Role: FOLLOWER
I20260619 18:47:43.782423 21260 ts_tablet_manager.cc:1428] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:43.782466 21259 consensus_queue.cc:260] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.782717 21259 ts_tablet_manager.cc:1428] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:43.782636 21266 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.782794 21266 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:43.783306 21264 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.783435 21264 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:43.834455 21264 raft_consensus.cc:482] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:43.834599 21264 raft_consensus.cc:504] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.835799 21264 leader_election.cc:288] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933), 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785)
I20260619 18:47:43.839856 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.836159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:46919 (local address 127.12.234.195:37785)
0619 18:47:43.836279 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.836296 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.836559 (+ 263us) server_negotiation.cc:407] Connection header received
0619 18:47:43.836823 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.836826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.836874 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.836960 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:43.837201 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.837793 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.838882 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.839257 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.839361 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.839415 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.839651 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.839720 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:47:43.839872 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.836058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:39933 (local address 127.12.234.193:39781)
0619 18:47:43.836412 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.836423 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.836535 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.836827 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.836829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.836846 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:43.836992 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.836999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.837735 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.837739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.838743 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.838753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.839383 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.839387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.839539 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.839562 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.839630 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.839735 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":64,"thread_start_us":106,"threads_started":1}
I20260619 18:47:43.839856 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.836105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:39781 (local address 127.12.234.194:39933)
0619 18:47:43.836250 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.836255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.836504 (+ 249us) server_negotiation.cc:407] Connection header received
0619 18:47:43.836660 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.836664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.836736 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.836809 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:43.837108 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.837634 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.838862 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.839267 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.839374 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.839426 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.839660 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.839730 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 18:47:43.839895 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.836067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37785 (local address 127.12.234.193:46919)
0619 18:47:43.836456 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.836467 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.836669 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.836978 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.836980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.836994 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:43.837093 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.837100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.837912 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.837915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.838760 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.838769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.839375 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.839378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.839518 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.839552 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.839618 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.839727 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":54,"thread_start_us":132,"threads_started":1}
I20260619 18:47:43.840533 21044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb4c344aeb14aa888e16249cc530eb4" candidate_uuid: "df41755a74c540d79683fecf6c4466ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32c71155665486dbadacafd9618f3ce" is_pre_election: true
I20260619 18:47:43.840778 21044 raft_consensus.cc:2454] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df41755a74c540d79683fecf6c4466ff in term 0.
I20260619 18:47:43.841195 20837 leader_election.cc:302] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32c71155665486dbadacafd9618f3ce, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:43.841316 21184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb4c344aeb14aa888e16249cc530eb4" candidate_uuid: "df41755a74c540d79683fecf6c4466ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9badb6e4da614640b2debf2778ba6521" is_pre_election: true
I20260619 18:47:43.841435 21264 raft_consensus.cc:2781] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:43.841459 21184 raft_consensus.cc:2454] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df41755a74c540d79683fecf6c4466ff in term 0.
I20260619 18:47:43.841495 21264 raft_consensus.cc:482] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:43.841539 21264 raft_consensus.cc:3037] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:43.842603 21264 raft_consensus.cc:504] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.842949 21264 leader_election.cc:288] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [CANDIDATE]: Term 1 election: Requested vote from peers b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933), 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785)
I20260619 18:47:43.843282 21184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb4c344aeb14aa888e16249cc530eb4" candidate_uuid: "df41755a74c540d79683fecf6c4466ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9badb6e4da614640b2debf2778ba6521"
I20260619 18:47:43.843322 21044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb4c344aeb14aa888e16249cc530eb4" candidate_uuid: "df41755a74c540d79683fecf6c4466ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32c71155665486dbadacafd9618f3ce"
I20260619 18:47:43.843391 21184 raft_consensus.cc:3037] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:43.843429 21044 raft_consensus.cc:3037] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:43.844300 21184 raft_consensus.cc:2454] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df41755a74c540d79683fecf6c4466ff in term 1.
I20260619 18:47:43.844398 21044 raft_consensus.cc:2454] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df41755a74c540d79683fecf6c4466ff in term 1.
I20260619 18:47:43.844578 20835 leader_election.cc:302] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9badb6e4da614640b2debf2778ba6521, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:43.844748 21264 raft_consensus.cc:2781] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:43.844961 21264 raft_consensus.cc:686] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [term 1 LEADER]: Becoming Leader. State: Replica: df41755a74c540d79683fecf6c4466ff, State: Running, Role: LEADER
I20260619 18:47:43.845207 21264 consensus_queue.cc:237] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:43.846629 21301 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: TxnStatusTablet state changed. Reason: New leader df41755a74c540d79683fecf6c4466ff. Latest consensus state: current_term: 1 leader_uuid: "df41755a74c540d79683fecf6c4466ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.846771 21301 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: This TxnStatusTablet replica's current role is: LEADER
I20260619 18:47:43.847045 21303 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 18:47:43.847203 21303 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 18:47:43.847801 20747 catalog_manager.cc:5495] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff reported cstate change: term changed from 0 to 1, leader changed from <none> to df41755a74c540d79683fecf6c4466ff (127.12.234.193). New cstate: current_term: 1 leader_uuid: "df41755a74c540d79683fecf6c4466ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260619 18:47:43.866215 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.862318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47282 (local address 127.12.234.193:34835)
0619 18:47:43.862432 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.862437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.862491 (+ 54us) server_negotiation.cc:407] Connection header received
0619 18:47:43.862951 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.862954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.863027 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.863112 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:43.863370 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.863908 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.864753 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.864971 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.865063 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.865356 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:43.865788 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:43.865893 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.865971 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.866037 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:47:43.866209 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.862261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34835 (local address 127.0.0.1:47282)
0619 18:47:43.862425 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.862441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.862759 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.863143 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.863146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.863154 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:43.863266 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.863273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.864021 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.864025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.864638 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.864647 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.865082 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.865086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.865179 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.865224 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:43.865883 (+ 659us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:43.865897 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.865960 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.866054 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 18:47:43.869638 21184 raft_consensus.cc:1261] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Refusing update from remote peer df41755a74c540d79683fecf6c4466ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:43.869876 21044 raft_consensus.cc:1261] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Refusing update from remote peer df41755a74c540d79683fecf6c4466ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:43.870232 21264 consensus_queue.cc:1035] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:43.870424 21301 consensus_queue.cc:1035] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:43.871932 21266 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: TxnStatusTablet state changed. Reason: New leader df41755a74c540d79683fecf6c4466ff. Latest consensus state: current_term: 1 leader_uuid: "df41755a74c540d79683fecf6c4466ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.872059 21266 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:43.872115 21265 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: TxnStatusTablet state changed. Reason: New leader df41755a74c540d79683fecf6c4466ff. Latest consensus state: current_term: 1 leader_uuid: "df41755a74c540d79683fecf6c4466ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.872226 21265 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:43.874147 21264 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df41755a74c540d79683fecf6c4466ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.874289 21264 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: This TxnStatusTablet replica's current role is: LEADER
I20260619 18:47:43.875531 21274 mvcc.cc:205] Tried to move back new op lower bound from 7298641362404929536 to 7298641362312548352. Current Snapshot: MvccSnapshot[applied={T|T < 7298641362404929536}]
I20260619 18:47:43.875730 21266 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df41755a74c540d79683fecf6c4466ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.875829 21266 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P 9badb6e4da614640b2debf2778ba6521: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:43.877714 21301 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df41755a74c540d79683fecf6c4466ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.877827 21301 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P df41755a74c540d79683fecf6c4466ff: This TxnStatusTablet replica's current role is: LEADER
I20260619 18:47:43.878779 21272 mvcc.cc:205] Tried to move back new op lower bound from 7298641362404929536 to 7298641362312548352. Current Snapshot: MvccSnapshot[applied={T|T < 7298641362404929536}]
I20260619 18:47:43.879585 21265 tablet_replica.cc:439] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df41755a74c540d79683fecf6c4466ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } } }
I20260619 18:47:43.879711 21265 tablet_replica.cc:442] T 8fb4c344aeb14aa888e16249cc530eb4 P b32c71155665486dbadacafd9618f3ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 18:47:43.889822 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.885824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34835 (local address 127.12.234.194:37041)
0619 18:47:43.886248 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.886261 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.886355 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.886596 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.886598 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.886605 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:43.886698 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.886702 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.887704 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.887707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.888379 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.888389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.888969 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.888972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.889057 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.889113 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:43.889489 (+ 376us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:43.889497 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.889599 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.889688 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":99,"threads_started":1}
I20260619 18:47:43.889827 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.885926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:37041 (local address 127.12.234.193:34835)
0619 18:47:43.886053 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.886058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.886327 (+ 269us) server_negotiation.cc:407] Connection header received
0619 18:47:43.886444 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.886447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.886503 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.886588 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:43.886796 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.887580 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.888494 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.888843 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.888946 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.889283 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:43.889378 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:43.889451 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.889622 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.889688 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:47:43.897403 21184 raft_consensus.cc:1261] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Refusing update from remote peer b32c71155665486dbadacafd9618f3ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:43.897485 20904 raft_consensus.cc:1261] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Refusing update from remote peer b32c71155665486dbadacafd9618f3ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:43.897836 21265 consensus_queue.cc:1035] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:43.898026 21277 consensus_queue.cc:1035] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:43.905355 21273 mvcc.cc:205] Tried to move back new op lower bound from 7298641362519724032 to 7298641361808519168. Current Snapshot: MvccSnapshot[applied={T|T < 7298641362519724032}]
I20260619 18:47:43.923729 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.919940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:39933 (local address 127.12.234.193:52087)
0619 18:47:43.920266 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:43.920280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:43.920383 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:43.920649 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:43.920652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:43.920659 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:43.920795 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.920802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.921664 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.921667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.922593 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:43.922603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.923032 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.923035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:43.923126 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.923173 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:43.923439 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:43.923445 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:47:43.923503 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 18:47:43.923592 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":111,"threads_started":1}
I20260619 18:47:43.923718 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.920014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:52087 (local address 127.12.234.194:39933)
0619 18:47:43.920211 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.920215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.920345 (+ 130us) server_negotiation.cc:407] Connection header received
0619 18:47:43.920516 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.920519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.920570 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.920633 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:43.920918 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.921540 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.922700 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.922922 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.923009 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.923296 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:43.923370 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:43.923425 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.923513 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.923569 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 18:47:43.950991 20747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:43.951169 20747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:43.953090 20747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: Sending DeleteTablet for 3 replicas of tablet 10418ed1508a4c4db9281ec6dd806db5
I20260619 18:47:43.953682 21023 tablet_service.cc:1514] Processing DeleteTablet for tablet 10418ed1508a4c4db9281ec6dd806db5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:43 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47594
I20260619 18:47:43.953706 20884 tablet_service.cc:1514] Processing DeleteTablet for tablet 10418ed1508a4c4db9281ec6dd806db5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:43 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47266
I20260619 18:47:43.953871 21283 tablet_replica.cc:330] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: stopping tablet replica
I20260619 18:47:43.953835 21164 tablet_service.cc:1514] Processing DeleteTablet for tablet 10418ed1508a4c4db9281ec6dd806db5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:43 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:39320
I20260619 18:47:43.953982 21283 raft_consensus.cc:2229] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:43.954001 21282 tablet_replica.cc:330] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: stopping tablet replica
I20260619 18:47:43.954093 21283 raft_consensus.cc:2258] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:43.954087 21282 raft_consensus.cc:2229] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:43.954218 21282 raft_consensus.cc:2258] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:43.954259 21284 tablet_replica.cc:330] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: stopping tablet replica
I20260619 18:47:43.954372 21284 raft_consensus.cc:2229] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:43.954552 21284 raft_consensus.cc:2258] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:43.955320 21282 ts_tablet_manager.cc:1905] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:43.955346 21283 ts_tablet_manager.cc:1905] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:43.957067 21284 ts_tablet_manager.cc:1905] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:43.958092 21283 ts_tablet_manager.cc:1918] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:43.958328 21282 ts_tablet_manager.cc:1918] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:43.958401 21282 log.cc:1196] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/10418ed1508a4c4db9281ec6dd806db5
I20260619 18:47:43.958631 21282 ts_tablet_manager.cc:1939] T 10418ed1508a4c4db9281ec6dd806db5 P 9badb6e4da614640b2debf2778ba6521: Deleting consensus metadata
I20260619 18:47:43.958159 21283 log.cc:1196] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/10418ed1508a4c4db9281ec6dd806db5
I20260619 18:47:43.958994 21283 ts_tablet_manager.cc:1939] T 10418ed1508a4c4db9281ec6dd806db5 P df41755a74c540d79683fecf6c4466ff: Deleting consensus metadata
I20260619 18:47:43.959456 20731 catalog_manager.cc:4841] TS 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785): tablet 10418ed1508a4c4db9281ec6dd806db5 (table test-table [id=3001911f5f1f4772b5ba529ebecfa754]) successfully deleted
I20260619 18:47:43.959794 21284 ts_tablet_manager.cc:1918] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:43.959776 20731 catalog_manager.cc:4841] TS df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835): tablet 10418ed1508a4c4db9281ec6dd806db5 (table test-table [id=3001911f5f1f4772b5ba529ebecfa754]) successfully deleted
I20260619 18:47:43.959857 21284 log.cc:1196] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/10418ed1508a4c4db9281ec6dd806db5
I20260619 18:47:43.960067 21284 ts_tablet_manager.cc:1939] T 10418ed1508a4c4db9281ec6dd806db5 P b32c71155665486dbadacafd9618f3ce: Deleting consensus metadata
I20260619 18:47:43.960633 20733 catalog_manager.cc:4841] TS b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933): tablet 10418ed1508a4c4db9281ec6dd806db5 (table test-table [id=3001911f5f1f4772b5ba529ebecfa754]) successfully deleted
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 18:47:43.983963 21295 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:43.977503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37330 (local address 127.12.234.254:44439)
0619 18:47:43.977656 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.977661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.977823 (+ 162us) server_negotiation.cc:407] Connection header received
0619 18:47:43.977967 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.977971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.978023 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.978095 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:43.979083 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.979680 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.980547 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.980777 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.980866 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.983836 (+ 2970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37330: BlockingRecv error: recv got EOF from 127.0.0.1:37330 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 18:47:43 dist-test-slave-fljv krb5kdc[20702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM
I20260619 18:47:43.993764 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:43.985606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37344 (local address 127.12.234.254:44439)
0619 18:47:43.985716 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:43.985729 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:43.986107 (+ 378us) server_negotiation.cc:407] Connection header received
0619 18:47:43.986260 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:43.986263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:43.986310 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:43.986374 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:43.986961 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.987489 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.988257 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:43.988484 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:43.988568 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:43.990963 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:43.990967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:43.990969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:43.991004 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:43.992282 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.992662 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.992666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.992667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.992742 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:43.992982 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:43.992985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:43.992987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:43.993162 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:43.993283 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:43.993501 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 18:47:43.993611 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:47:43.996553 20747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:43.997007 20747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:44.001986 20884 tablet_service.cc:1467] Processing CreateTablet for tablet 22e611a84ca94af9af5bc7201764b406 (DEFAULT_TABLE table=test-table [id=709461ba756e4005929e536b2812c1ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:44.002151 21164 tablet_service.cc:1467] Processing CreateTablet for tablet 22e611a84ca94af9af5bc7201764b406 (DEFAULT_TABLE table=test-table [id=709461ba756e4005929e536b2812c1ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:44.002331 20884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e611a84ca94af9af5bc7201764b406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:44.002468 21164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e611a84ca94af9af5bc7201764b406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:44.004423 21260 tablet_bootstrap.cc:492] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: Bootstrap starting.
I20260619 18:47:44.004457 21259 tablet_bootstrap.cc:492] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: Bootstrap starting.
I20260619 18:47:44.004824 21023 tablet_service.cc:1467] Processing CreateTablet for tablet 22e611a84ca94af9af5bc7201764b406 (DEFAULT_TABLE table=test-table [id=709461ba756e4005929e536b2812c1ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:44.005101 21023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e611a84ca94af9af5bc7201764b406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:44.005395 21260 tablet_bootstrap.cc:654] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:44.005393 21259 tablet_bootstrap.cc:654] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:44.007786 21259 tablet_bootstrap.cc:492] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: No bootstrap required, opened a new log
I20260619 18:47:44.007895 21259 ts_tablet_manager.cc:1397] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:44.008354 21259 raft_consensus.cc:348] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.008471 21259 raft_consensus.cc:374] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:44.008498 21259 raft_consensus.cc:729] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df41755a74c540d79683fecf6c4466ff, State: Initialized, Role: FOLLOWER
I20260619 18:47:44.008610 21259 consensus_queue.cc:260] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.008862 21259 ts_tablet_manager.cc:1428] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:44.008881 21260 tablet_bootstrap.cc:492] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: No bootstrap required, opened a new log
I20260619 18:47:44.008965 21260 ts_tablet_manager.cc:1397] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 18:47:44.009084 21258 tablet_bootstrap.cc:492] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: Bootstrap starting.
I20260619 18:47:44.009418 21260 raft_consensus.cc:348] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.009518 21260 raft_consensus.cc:374] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:44.009548 21260 raft_consensus.cc:729] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9badb6e4da614640b2debf2778ba6521, State: Initialized, Role: FOLLOWER
I20260619 18:47:44.009680 21260 consensus_queue.cc:260] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.009912 21260 ts_tablet_manager.cc:1428] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 18:47:44.009910 21258 tablet_bootstrap.cc:654] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:44.011125 21258 tablet_bootstrap.cc:492] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: No bootstrap required, opened a new log
I20260619 18:47:44.011215 21258 ts_tablet_manager.cc:1397] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:44.011581 21258 raft_consensus.cc:348] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.011706 21258 raft_consensus.cc:374] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:44.011744 21258 raft_consensus.cc:729] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32c71155665486dbadacafd9618f3ce, State: Initialized, Role: FOLLOWER
I20260619 18:47:44.011868 21258 consensus_queue.cc:260] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.012112 21258 ts_tablet_manager.cc:1428] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:44.027788 21264 raft_consensus.cc:482] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:44.027917 21264 raft_consensus.cc:504] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.028316 21264 leader_election.cc:288] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933), 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785)
I20260619 18:47:44.028705 21044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22e611a84ca94af9af5bc7201764b406" candidate_uuid: "df41755a74c540d79683fecf6c4466ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32c71155665486dbadacafd9618f3ce" is_pre_election: true
I20260619 18:47:44.028731 21184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22e611a84ca94af9af5bc7201764b406" candidate_uuid: "df41755a74c540d79683fecf6c4466ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9badb6e4da614640b2debf2778ba6521" is_pre_election: true
I20260619 18:47:44.028842 21044 raft_consensus.cc:2454] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df41755a74c540d79683fecf6c4466ff in term 0.
I20260619 18:47:44.028888 21184 raft_consensus.cc:2454] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df41755a74c540d79683fecf6c4466ff in term 0.
I20260619 18:47:44.029173 20837 leader_election.cc:302] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9badb6e4da614640b2debf2778ba6521, b32c71155665486dbadacafd9618f3ce, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:44.029357 21264 raft_consensus.cc:2781] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:44.029415 21264 raft_consensus.cc:482] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:44.029440 21264 raft_consensus.cc:3037] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:44.030462 21264 raft_consensus.cc:504] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.030794 21264 leader_election.cc:288] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [CANDIDATE]: Term 1 election: Requested vote from peers b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933), 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785)
I20260619 18:47:44.031141 21184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22e611a84ca94af9af5bc7201764b406" candidate_uuid: "df41755a74c540d79683fecf6c4466ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9badb6e4da614640b2debf2778ba6521"
I20260619 18:47:44.031126 21044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22e611a84ca94af9af5bc7201764b406" candidate_uuid: "df41755a74c540d79683fecf6c4466ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32c71155665486dbadacafd9618f3ce"
I20260619 18:47:44.031255 21184 raft_consensus.cc:3037] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:44.031262 21044 raft_consensus.cc:3037] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:44.032246 21184 raft_consensus.cc:2454] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df41755a74c540d79683fecf6c4466ff in term 1.
I20260619 18:47:44.032241 21044 raft_consensus.cc:2454] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df41755a74c540d79683fecf6c4466ff in term 1.
I20260619 18:47:44.032573 20837 leader_election.cc:302] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32c71155665486dbadacafd9618f3ce, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:44.032719 21264 raft_consensus.cc:2781] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:44.032802 21264 raft_consensus.cc:686] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 1 LEADER]: Becoming Leader. State: Replica: df41755a74c540d79683fecf6c4466ff, State: Running, Role: LEADER
I20260619 18:47:44.032936 21264 consensus_queue.cc:237] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.034581 20746 catalog_manager.cc:5495] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff reported cstate change: term changed from 0 to 1, leader changed from <none> to df41755a74c540d79683fecf6c4466ff (127.12.234.193). New cstate: current_term: 1 leader_uuid: "df41755a74c540d79683fecf6c4466ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260619 18:47:44.046622 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.043620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47290 (local address 127.12.234.193:34835)
0619 18:47:44.043736 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:44.043740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:44.043753 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:47:44.043867 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:44.043869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:44.043916 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:44.043979 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:44.044239 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.044744 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.045660 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.045867 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.045953 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.046211 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:44.046290 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:44.046344 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:44.046459 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 18:47:44.046511 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 18:47:44.048926 21184 raft_consensus.cc:1261] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Refusing update from remote peer df41755a74c540d79683fecf6c4466ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:44.048926 21044 raft_consensus.cc:1261] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Refusing update from remote peer df41755a74c540d79683fecf6c4466ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:44.049393 21301 consensus_queue.cc:1035] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:44.049548 21264 consensus_queue.cc:1035] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:44.058243 20745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:44.058406 20745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:44.060475 20745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: Sending DeleteTablet for 3 replicas of tablet 22e611a84ca94af9af5bc7201764b406
I20260619 18:47:44.061097 21023 tablet_service.cc:1514] Processing DeleteTablet for tablet 22e611a84ca94af9af5bc7201764b406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:44 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47594
I20260619 18:47:44.061097 21164 tablet_service.cc:1514] Processing DeleteTablet for tablet 22e611a84ca94af9af5bc7201764b406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:44 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:39320
I20260619 18:47:44.061177 20884 tablet_service.cc:1514] Processing DeleteTablet for tablet 22e611a84ca94af9af5bc7201764b406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:44 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47266
I20260619 18:47:44.061283 21284 tablet_replica.cc:330] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: stopping tablet replica
I20260619 18:47:44.061283 21282 tablet_replica.cc:330] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: stopping tablet replica
I20260619 18:47:44.061393 21282 raft_consensus.cc:2229] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:44.061394 21284 raft_consensus.cc:2229] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:44.061512 21284 raft_consensus.cc:2258] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:44.061522 21282 raft_consensus.cc:2258] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:44.061695 21283 tablet_replica.cc:330] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: stopping tablet replica
I20260619 18:47:44.061808 21283 raft_consensus.cc:2229] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:44.062144 21283 raft_consensus.cc:2258] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:44.063309 21282 ts_tablet_manager.cc:1905] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:44.063349 21283 ts_tablet_manager.cc:1905] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:44.063239 20745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:47:44.063710 20745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:47:44.065757 21283 ts_tablet_manager.cc:1918] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:44.065757 21282 ts_tablet_manager.cc:1918] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:44.065829 21283 log.cc:1196] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/22e611a84ca94af9af5bc7201764b406
I20260619 18:47:44.065829 21282 log.cc:1196] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/22e611a84ca94af9af5bc7201764b406
I20260619 18:47:44.066062 21282 ts_tablet_manager.cc:1939] T 22e611a84ca94af9af5bc7201764b406 P 9badb6e4da614640b2debf2778ba6521: Deleting consensus metadata
I20260619 18:47:44.066062 21283 ts_tablet_manager.cc:1939] T 22e611a84ca94af9af5bc7201764b406 P df41755a74c540d79683fecf6c4466ff: Deleting consensus metadata
I20260619 18:47:44.066932 20731 catalog_manager.cc:4841] TS 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785): tablet 22e611a84ca94af9af5bc7201764b406 (table test-table [id=709461ba756e4005929e536b2812c1ec]) successfully deleted
I20260619 18:47:44.067328 20731 catalog_manager.cc:4841] TS df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835): tablet 22e611a84ca94af9af5bc7201764b406 (table test-table [id=709461ba756e4005929e536b2812c1ec]) successfully deleted
I20260619 18:47:44.068140 21284 ts_tablet_manager.cc:1905] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:44.070493 21284 ts_tablet_manager.cc:1918] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:47:44.070569 21284 log.cc:1196] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/22e611a84ca94af9af5bc7201764b406
I20260619 18:47:44.070793 21284 ts_tablet_manager.cc:1939] T 22e611a84ca94af9af5bc7201764b406 P b32c71155665486dbadacafd9618f3ce: Deleting consensus metadata
I20260619 18:47:44.071605 20733 catalog_manager.cc:4841] TS b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933): tablet 22e611a84ca94af9af5bc7201764b406 (table test-table [id=709461ba756e4005929e536b2812c1ec]) successfully deleted
I20260619 18:47:44.071949 21023 tablet_service.cc:1467] Processing CreateTablet for tablet c36170c3cbd341059514f017141c571f (DEFAULT_TABLE table=test-table [id=047c39546d5d4a73a0e8b82f215550c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:44.072211 21023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36170c3cbd341059514f017141c571f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:44.072166 20884 tablet_service.cc:1467] Processing CreateTablet for tablet c36170c3cbd341059514f017141c571f (DEFAULT_TABLE table=test-table [id=047c39546d5d4a73a0e8b82f215550c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:44.072301 21164 tablet_service.cc:1467] Processing CreateTablet for tablet c36170c3cbd341059514f017141c571f (DEFAULT_TABLE table=test-table [id=047c39546d5d4a73a0e8b82f215550c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:47:44.072402 20884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36170c3cbd341059514f017141c571f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:44.072567 21164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36170c3cbd341059514f017141c571f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:44.074712 21259 tablet_bootstrap.cc:492] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff: Bootstrap starting.
I20260619 18:47:44.074860 21260 tablet_bootstrap.cc:492] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: Bootstrap starting.
I20260619 18:47:44.074934 21258 tablet_bootstrap.cc:492] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: Bootstrap starting.
I20260619 18:47:44.075630 21259 tablet_bootstrap.cc:654] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:44.075758 21258 tablet_bootstrap.cc:654] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:44.076089 21260 tablet_bootstrap.cc:654] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:44.077231 21259 tablet_bootstrap.cc:492] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff: No bootstrap required, opened a new log
I20260619 18:47:44.077318 21259 ts_tablet_manager.cc:1397] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:44.077395 21260 tablet_bootstrap.cc:492] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: No bootstrap required, opened a new log
I20260619 18:47:44.077466 21260 ts_tablet_manager.cc:1397] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 18:47:44.077665 21258 tablet_bootstrap.cc:492] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: No bootstrap required, opened a new log
I20260619 18:47:44.077740 21258 ts_tablet_manager.cc:1397] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:44.077751 21259 raft_consensus.cc:348] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.077862 21259 raft_consensus.cc:374] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:44.077890 21259 raft_consensus.cc:729] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df41755a74c540d79683fecf6c4466ff, State: Initialized, Role: FOLLOWER
I20260619 18:47:44.077837 21260 raft_consensus.cc:348] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.077927 21260 raft_consensus.cc:374] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:44.077952 21260 raft_consensus.cc:729] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9badb6e4da614640b2debf2778ba6521, State: Initialized, Role: FOLLOWER
I20260619 18:47:44.078008 21259 consensus_queue.cc:260] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.078061 21260 consensus_queue.cc:260] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.078101 21258 raft_consensus.cc:348] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.078233 21258 raft_consensus.cc:374] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:44.078266 21258 raft_consensus.cc:729] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32c71155665486dbadacafd9618f3ce, State: Initialized, Role: FOLLOWER
I20260619 18:47:44.078320 21260 ts_tablet_manager.cc:1428] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:44.078382 21258 consensus_queue.cc:260] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.078622 21258 ts_tablet_manager.cc:1428] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 18:47:44.079970 21259 ts_tablet_manager.cc:1428] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:44.088881 21266 raft_consensus.cc:482] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:47:44.089039 21266 raft_consensus.cc:504] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.090365 21266 leader_election.cc:288] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835), b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933)
I20260619 18:47:44.094866 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.090729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:53637 (local address 127.12.234.193:34835)
0619 18:47:44.090849 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:44.090853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:44.091081 (+ 228us) server_negotiation.cc:407] Connection header received
0619 18:47:44.091281 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:44.091284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:44.091331 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:44.091394 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:44.091765 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.092487 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.093593 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.094098 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.094252 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.094307 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:44.094637 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 18:47:44.094734 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:47:44.095232 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.090932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:40519 (local address 127.12.234.194:39933)
0619 18:47:44.091055 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:44.091060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:44.091074 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:44.091198 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:44.091201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:44.091253 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:44.091318 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:47:44.091570 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.092365 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.094008 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.094605 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.094705 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.094748 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:44.095025 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 18:47:44.095107 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 18:47:44.095314 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.090556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34835 (local address 127.12.234.195:53637)
0619 18:47:44.090997 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:44.091011 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:44.091155 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:44.091496 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:44.091499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:44.091515 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:44.091631 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:44.091648 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.092605 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.092608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:44.093476 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:44.093486 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.094383 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.094387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:44.094482 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.094506 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 18:47:44.095094 (+ 588us) client_negotiation.cc:240] Negotiation successful
0619 18:47:44.095165 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":20,"thread_start_us":79,"threads_started":1}
I20260619 18:47:44.095631 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.090556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:39933 (local address 127.12.234.195:40519)
0619 18:47:44.090985 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:44.090998 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:44.091096 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:44.091337 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:44.091339 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:44.091354 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:47:44.091474 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:44.091479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.092478 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.092481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:44.093850 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:44.093860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.094804 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.094807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:44.094891 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.094913 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:47:44.095432 (+ 519us) client_negotiation.cc:240] Negotiation successful
0619 18:47:44.095505 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":42,"thread_start_us":95,"threads_started":1}
I20260619 18:47:44.095744 20904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36170c3cbd341059514f017141c571f" candidate_uuid: "9badb6e4da614640b2debf2778ba6521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df41755a74c540d79683fecf6c4466ff" is_pre_election: true
I20260619 18:47:44.095892 20904 raft_consensus.cc:2454] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9badb6e4da614640b2debf2778ba6521 in term 0.
I20260619 18:47:44.096019 21044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36170c3cbd341059514f017141c571f" candidate_uuid: "9badb6e4da614640b2debf2778ba6521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32c71155665486dbadacafd9618f3ce" is_pre_election: true
I20260619 18:47:44.096154 21044 raft_consensus.cc:2454] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9badb6e4da614640b2debf2778ba6521 in term 0.
I20260619 18:47:44.096288 21115 leader_election.cc:302] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9badb6e4da614640b2debf2778ba6521, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:44.096601 21266 raft_consensus.cc:2781] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:47:44.096655 21266 raft_consensus.cc:482] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:47:44.096692 21266 raft_consensus.cc:3037] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:44.097664 21266 raft_consensus.cc:504] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.098006 21266 leader_election.cc:288] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [CANDIDATE]: Term 1 election: Requested vote from peers df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835), b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933)
I20260619 18:47:44.098346 21044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36170c3cbd341059514f017141c571f" candidate_uuid: "9badb6e4da614640b2debf2778ba6521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32c71155665486dbadacafd9618f3ce"
I20260619 18:47:44.098337 20904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36170c3cbd341059514f017141c571f" candidate_uuid: "9badb6e4da614640b2debf2778ba6521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df41755a74c540d79683fecf6c4466ff"
I20260619 18:47:44.098459 20904 raft_consensus.cc:3037] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:44.098459 21044 raft_consensus.cc:3037] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:44.099516 20904 raft_consensus.cc:2454] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9badb6e4da614640b2debf2778ba6521 in term 1.
I20260619 18:47:44.099515 21044 raft_consensus.cc:2454] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9badb6e4da614640b2debf2778ba6521 in term 1.
I20260619 18:47:44.099864 21117 leader_election.cc:302] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9badb6e4da614640b2debf2778ba6521, b32c71155665486dbadacafd9618f3ce, df41755a74c540d79683fecf6c4466ff; no voters:
I20260619 18:47:44.100075 21266 raft_consensus.cc:2781] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:44.100351 21266 raft_consensus.cc:686] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 1 LEADER]: Becoming Leader. State: Replica: 9badb6e4da614640b2debf2778ba6521, State: Running, Role: LEADER
I20260619 18:47:44.100618 21266 consensus_queue.cc:237] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } }
I20260619 18:47:44.103179 20743 catalog_manager.cc:5495] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9badb6e4da614640b2debf2778ba6521 (127.12.234.195). New cstate: current_term: 1 leader_uuid: "9badb6e4da614640b2debf2778ba6521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9badb6e4da614640b2debf2778ba6521" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37785 } health_report { overall_health: HEALTHY } } }
I20260619 18:47:44.120548 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.117069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39324 (local address 127.12.234.195:37785)
0619 18:47:44.117184 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:44.117188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:44.117201 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:47:44.117374 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:44.117377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:44.117432 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:44.117522 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:44.117770 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.118386 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.119187 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.119399 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.119483 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.119731 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:44.120187 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:44.120288 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:44.120355 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 18:47:44.120411 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 18:47:44.131385 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.128165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:50287 (local address 127.12.234.193:34835)
0619 18:47:44.128284 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:44.128288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:44.128411 (+ 123us) server_negotiation.cc:407] Connection header received
0619 18:47:44.128544 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:44.128547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:44.128598 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:44.128660 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:44.128894 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.129440 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.130307 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.130542 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.130626 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.130933 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:44.131021 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:44.131079 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:44.131188 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 18:47:44.131239 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 18:47:44.131399 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.128033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34835 (local address 127.12.234.195:50287)
0619 18:47:44.128335 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:44.128347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:44.128450 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:44.128677 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:44.128680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:44.128688 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:44.128804 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:44.128809 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.129543 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.129546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:44.130211 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:44.130221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.130640 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.130644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:44.130739 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.130782 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:44.131093 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:44.131100 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 18:47:44.131158 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 18:47:44.131239 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":107,"threads_started":1}
I20260619 18:47:44.137241 21044 raft_consensus.cc:1261] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Refusing update from remote peer 9badb6e4da614640b2debf2778ba6521: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:44.137406 20904 raft_consensus.cc:1261] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Refusing update from remote peer 9badb6e4da614640b2debf2778ba6521: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:47:44.137935 21266 consensus_queue.cc:1035] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df41755a74c540d79683fecf6c4466ff" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:44.138115 21341 consensus_queue.cc:1035] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32c71155665486dbadacafd9618f3ce" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:47:44.158840 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.154750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37785 (local address 127.12.234.193:52689)
0619 18:47:44.155006 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:44.155021 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:44.155102 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:44.155642 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:44.155644 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:44.155652 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 18:47:44.155761 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:44.155768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.156451 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.156455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:44.157465 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:44.157476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.158003 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.158006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:44.158171 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.158242 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 18:47:44.158553 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 18:47:44.158560 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 18:47:44.158653 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 18:47:44.158735 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260619 18:47:44.159157 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.155258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:52689 (local address 127.12.234.195:37785)
0619 18:47:44.155362 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:44.155366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:44.155379 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:47:44.155418 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:44.155421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:44.155468 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:44.155533 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:47:44.155854 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.156350 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.157599 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:44.157912 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:44.158005 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:44.158375 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:47:44.158461 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:47:44.158526 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:44.158990 (+ 464us) server_negotiation.cc:299] Negotiation successful
0619 18:47:44.159046 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:47:44.183714 20743 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:44.183898 20743 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:47:44.189281 20743 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b076ecefbbf429d925e3e7da9b72351: Sending DeleteTablet for 3 replicas of tablet c36170c3cbd341059514f017141c571f
I20260619 18:47:44.190106 21023 tablet_service.cc:1514] Processing DeleteTablet for tablet c36170c3cbd341059514f017141c571f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:44 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47594
I20260619 18:47:44.190249 21164 tablet_service.cc:1514] Processing DeleteTablet for tablet c36170c3cbd341059514f017141c571f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:44 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:39320
I20260619 18:47:44.190331 21284 tablet_replica.cc:330] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: stopping tablet replica
I20260619 18:47:44.190441 21284 raft_consensus.cc:2229] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:44.190582 21284 raft_consensus.cc:2258] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:44.190644 20884 tablet_service.cc:1514] Processing DeleteTablet for tablet c36170c3cbd341059514f017141c571f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:47:44 UTC) from {username='oryx', principal='oryx/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47266
I20260619 18:47:44.190739 21282 tablet_replica.cc:330] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: stopping tablet replica
I20260619 18:47:44.190837 21282 raft_consensus.cc:2229] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:47:44.190860 21283 tablet_replica.cc:330] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff: stopping tablet replica
I20260619 18:47:44.190976 21283 raft_consensus.cc:2229] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:47:44.191087 21283 raft_consensus.cc:2258] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:44.191159 21282 raft_consensus.cc:2258] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:47:44.191886 21283 ts_tablet_manager.cc:1905] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:44.192032 21284 ts_tablet_manager.cc:1905] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:44.194548 21284 ts_tablet_manager.cc:1918] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:44.194628 21284 log.cc:1196] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/c36170c3cbd341059514f017141c571f
I20260619 18:47:44.194852 21284 ts_tablet_manager.cc:1939] T c36170c3cbd341059514f017141c571f P b32c71155665486dbadacafd9618f3ce: Deleting consensus metadata
I20260619 18:47:44.195497 21283 ts_tablet_manager.cc:1918] T c36170c3cbd341059514f017141c571f P df41755a74c540d79683fecf6c4466ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:44.195605 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 20819
I20260619 18:47:44.198364 20733 catalog_manager.cc:4841] TS b32c71155665486dbadacafd9618f3ce (127.12.234.194:39933): tablet c36170c3cbd341059514f017141c571f (table test-table [id=047c39546d5d4a73a0e8b82f215550c8]) successfully deleted
I20260619 18:47:44.200984 21282 ts_tablet_manager.cc:1905] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:47:44.203763 21282 ts_tablet_manager.cc:1918] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 18:47:44.203858 21282 log.cc:1196] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/c36170c3cbd341059514f017141c571f
I20260619 18:47:44.204118 21282 ts_tablet_manager.cc:1939] T c36170c3cbd341059514f017141c571f P 9badb6e4da614640b2debf2778ba6521: Deleting consensus metadata
I20260619 18:47:44.205147 20731 catalog_manager.cc:4841] TS 9badb6e4da614640b2debf2778ba6521 (127.12.234.195:37785): tablet c36170c3cbd341059514f017141c571f (table test-table [id=047c39546d5d4a73a0e8b82f215550c8]) successfully deleted
W20260619 18:47:44.209012 20731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:34835 (error 108)
I20260619 18:47:44.209300 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 20959
I20260619 18:47:44.214583 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:44.210904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:34835 (local address 127.0.0.1:47294)
0619 18:47:44.214391 (+ 3487us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:44.214478 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:34835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3395,"thread_start_us":64,"threads_started":1}
W20260619 18:47:44.214833 20731 catalog_manager.cc:4584] TS df41755a74c540d79683fecf6c4466ff (127.12.234.193:34835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c36170c3cbd341059514f017141c571f: Network error: Client connection negotiation failed: client connection to 127.12.234.193:34835: connect: Connection refused (error 111)
I20260619 18:47:44.218361 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 21099
I20260619 18:47:44.223696 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 20715
2026-06-19T18:47:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3027 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:44 dist-test-slave-fljv krb5kdc[21355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:44 dist-test-slave-fljv krb5kdc[21355](info): set up 2 sockets
Jun 19 18:47:44 dist-test-slave-fljv krb5kdc[21355](info): commencing operation
krb5kdc: starting...
W20260619 18:47:45.202657 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:45 dist-test-slave-fljv krb5kdc[21355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:45.327340 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40337
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:40337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:45.435871 21368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:45.436165 21368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:45.436208 21368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:45.440315 21368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:45.440397 21368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:45.440414 21368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:45.440430 21368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:45.440444 21368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:45.446105 21368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:40337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:45.447300 21368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:45.448351 21368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:45.457015 21373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:45.457088 21368 server_base.cc:1034] running on GCE node
W20260619 18:47:45.457029 21374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:45.457029 21376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:45.457765 21368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:45.459002 21368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:45.460203 21368 hybrid_clock.cc:648] HybridClock initialized: now 1781894865460162 us; error 54 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T18:47:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1241 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:47:45 dist-test-slave-fljv krb5kdc[21381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:47:45 dist-test-slave-fljv krb5kdc[21381](info): set up 2 sockets
Jun 19 18:47:45 dist-test-slave-fljv krb5kdc[21381](info): commencing operation
krb5kdc: starting...
W20260619 18:47:46.422626 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:47:46 dist-test-slave-fljv krb5kdc[21381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:47:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:47:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:47:46.538702 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40337
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:40337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:46.647568 21394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:46.647866 21394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:46.647922 21394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:46.651961 21394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:46.652031 21394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:46.652056 21394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:46.652078 21394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:46.652107 21394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:46.657469 21394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:40337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:46.658627 21394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:46.659648 21394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:46.668416 21400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:46.668471 21399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:46.668447 21394 server_base.cc:1034] running on GCE node
W20260619 18:47:46.668555 21402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:46.669337 21394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:46.670542 21394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:46.671706 21394 hybrid_clock.cc:648] HybridClock initialized: now 1781894866671676 us; error 42 us; skew 500 ppm
Jun 19 18:47:46 dist-test-slave-fljv krb5kdc[21381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894866, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:46.674476 21394 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:46.675539 21394 webserver.cc:460] Webserver started at http://127.12.234.254:35477/ using document root <none> and password file <none>
I20260619 18:47:46.676122 21394 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:46.676191 21394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:46.676390 21394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:46.678535 21394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "b0f9c0f343b14502b6e3c89c499f6f21"
format_stamp: "Formatted at 2026-06-19 18:47:46 on dist-test-slave-fljv"
server_key: "dfc66debe7d852be4257de9b1564d36a"
server_key_iv: "dfe741b75084477fc8872e493edf56cc"
server_key_version: "encryptionkey@0"
I20260619 18:47:46.679116 21394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "b0f9c0f343b14502b6e3c89c499f6f21"
format_stamp: "Formatted at 2026-06-19 18:47:46 on dist-test-slave-fljv"
server_key: "dfc66debe7d852be4257de9b1564d36a"
server_key_iv: "dfe741b75084477fc8872e493edf56cc"
server_key_version: "encryptionkey@0"
I20260619 18:47:46.682874 21394 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:47:46.685276 21409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:46.686559 21394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:46.686669 21394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "b0f9c0f343b14502b6e3c89c499f6f21"
format_stamp: "Formatted at 2026-06-19 18:47:46 on dist-test-slave-fljv"
server_key: "dfc66debe7d852be4257de9b1564d36a"
server_key_iv: "dfe741b75084477fc8872e493edf56cc"
server_key_version: "encryptionkey@0"
I20260619 18:47:46.686779 21394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:46.698590 21394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:46.701735 21394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:46.701918 21394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:46.709017 21394 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:40337
I20260619 18:47:46.709038 21461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:40337 every 8 connection(s)
I20260619 18:47:46.710035 21394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:46.712934 21462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:47:46.714464 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 21394
I20260619 18:47:46.714589 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:47:46.714857 13227 external_mini_cluster.cc:1428] Setting key f5ec47c1cdf27894687df4b13f4ef940
I20260619 18:47:46.719030 21462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: Bootstrap starting.
I20260619 18:47:46.721474 21462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: Neither blocks nor log segments found. Creating new log.
I20260619 18:47:46.722342 21462 log.cc:824] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: Log is configured to *not* fsync() on all Append() calls
Jun 19 18:47:46 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:46.724259 21462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: No bootstrap required, opened a new log
I20260619 18:47:46.727775 21462 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } }
I20260619 18:47:46.727999 21462 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:47:46.728044 21462 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f9c0f343b14502b6e3c89c499f6f21, State: Initialized, Role: FOLLOWER
I20260619 18:47:46.728493 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:46.716240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33116 (local address 127.12.234.254:40337)
0619 18:47:46.716647 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:46.716662 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:46.716698 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:47:46.717310 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:46.717327 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:46.717668 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:46.717955 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:46.718845 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:46.719824 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:46.720971 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:46.721479 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:46.721646 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:46.724159 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:46.724164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:46.724178 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:46.724204 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:46.725763 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:46.726161 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:46.726166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:46.726171 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:46.726305 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:46.726552 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:46.726555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:46.726557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:46.726959 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:46.727150 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:46.727426 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 18:47:46.727710 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":123,"threads_started":1}
I20260619 18:47:46.728475 21462 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } }
I20260619 18:47:46.728595 21462 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:46.728644 21462 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:46.728735 21462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:47:46.729568 21462 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } }
I20260619 18:47:46.729882 21462 leader_election.cc:302] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0f9c0f343b14502b6e3c89c499f6f21; no voters:
I20260619 18:47:46.730165 21462 leader_election.cc:288] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:47:46.730274 21467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:47:46.730429 21467 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 1 LEADER]: Becoming Leader. State: Replica: b0f9c0f343b14502b6e3c89c499f6f21, State: Running, Role: LEADER
I20260619 18:47:46.730731 21467 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } }
I20260619 18:47:46.731446 21462 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:46.731894 21467 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } } }
I20260619 18:47:46.732009 21467 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:46.732225 21467 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0f9c0f343b14502b6e3c89c499f6f21. Latest consensus state: current_term: 1 leader_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } } }
I20260619 18:47:46.732374 21467 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:46.732390 21472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:46.735445 21472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:46.741124 21472 catalog_manager.cc:1347] Generated new cluster ID: 3177310961db4ef3bde7adfb0b14690b
I20260619 18:47:46.741191 21472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:46.757725 21472 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:47:46.759083 21472 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:46.771940 21472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: Generated new TSK 0
I20260619 18:47:46.772895 21472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:47:46.801757 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40337
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:46.910414 21488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:46.910681 21488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:46.910723 21488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:46.914688 21488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:46.914752 21488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:46.914812 21488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:46.920140 21488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:46.921175 21488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:46.922148 21488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:46.931659 21496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:46.931696 21493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:46.931666 21494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:46.931939 21488 server_base.cc:1034] running on GCE node
I20260619 18:47:46.932514 21488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:46.933202 21488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:46.934381 21488 hybrid_clock.cc:648] HybridClock initialized: now 1781894866934330 us; error 73 us; skew 500 ppm
Jun 19 18:47:46 dist-test-slave-fljv krb5kdc[21381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894866, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:46.937178 21488 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:47:46.938155 21488 webserver.cc:460] Webserver started at http://127.12.234.193:45347/ using document root <none> and password file <none>
I20260619 18:47:46.938745 21488 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:46.938810 21488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:46.939013 21488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:46.941107 21488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "96214954997d4016b67f15731bbbd6d9"
format_stamp: "Formatted at 2026-06-19 18:47:46 on dist-test-slave-fljv"
server_key: "8f1e28302f987fddddaac129ad2eacda"
server_key_iv: "41b6f06246a85e2c30909bd2600eb6cb"
server_key_version: "encryptionkey@0"
I20260619 18:47:46.941637 21488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "96214954997d4016b67f15731bbbd6d9"
format_stamp: "Formatted at 2026-06-19 18:47:46 on dist-test-slave-fljv"
server_key: "8f1e28302f987fddddaac129ad2eacda"
server_key_iv: "41b6f06246a85e2c30909bd2600eb6cb"
server_key_version: "encryptionkey@0"
I20260619 18:47:46.945416 21488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:46.947903 21503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:46.949095 21488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 18:47:46.949201 21488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "96214954997d4016b67f15731bbbd6d9"
format_stamp: "Formatted at 2026-06-19 18:47:46 on dist-test-slave-fljv"
server_key: "8f1e28302f987fddddaac129ad2eacda"
server_key_iv: "41b6f06246a85e2c30909bd2600eb6cb"
server_key_version: "encryptionkey@0"
I20260619 18:47:46.949294 21488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:46.973186 21488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:46.976513 21488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:46.976704 21488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:46.977303 21488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:46.978154 21488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:46.978232 21488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:46.978287 21488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:46.978310 21488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:46.986670 21488 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:42071
I20260619 18:47:46.986701 21616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:42071 every 8 connection(s)
I20260619 18:47:46.987608 21488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:46.987958 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 21488
I20260619 18:47:46.988088 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:47:46.988417 13227 external_mini_cluster.cc:1428] Setting key a534021a05b255f7f780eb03870486f0
Jun 19 18:47:46 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894866, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:47:47.002715 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:46.989499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:56051 (local address 127.12.234.254:40337)
0619 18:47:46.989670 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:46.989675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:46.990709 (+ 1034us) server_negotiation.cc:407] Connection header received
0619 18:47:46.992248 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:46.992251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:46.992299 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:46.992367 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:46.993622 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:46.994216 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:46.995911 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:46.996285 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:46.996416 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:46.999126 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:46.999130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:46.999132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:46.999159 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:47.000488 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:47.001038 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:47.001041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:47.001042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:47.001124 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:47.001471 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:47.001474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:47.001475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:47.001743 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:47.001845 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:47.002461 (+ 616us) server_negotiation.cc:299] Negotiation successful
0619 18:47:47.002579 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 18:47:47.003844 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:46.990032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:56051)
0619 18:47:46.990534 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:46.990587 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:46.991995 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:46.992591 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:46.992600 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:46.992971 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:46.993470 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:46.993483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:46.994361 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:46.994364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:46.995759 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:46.995771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:46.996467 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:46.996470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:46.996636 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:46.997252 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:46.997269 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:46.998941 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:47.000608 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:47.000613 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:47.000629 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:47.000923 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:47.001201 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:47.001204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:47.001205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:47.001341 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:47.001843 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:47.001849 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:47.002171 (+ 322us) client_negotiation.cc:769] Sending connection context
0619 18:47:47.002416 (+ 245us) client_negotiation.cc:240] Negotiation successful
0619 18:47:47.002821 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":130,"threads_started":1}
I20260619 18:47:47.005088 21617 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40337
I20260619 18:47:47.005327 21617 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:47.005854 21617 heartbeater.cc:511] Master 127.12.234.254:40337 requested a full tablet report, sending...
I20260619 18:47:47.007457 21426 ts_manager.cc:194] Registered new tserver with Master: 96214954997d4016b67f15731bbbd6d9 (127.12.234.193:42071)
I20260619 18:47:47.008822 21426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:56051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:47:47.018314 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40337
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:47.125720 21622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:47.125999 21622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:47.126058 21622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:47.130067 21622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:47.130131 21622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:47.130232 21622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:47.135667 21622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:47.136716 21622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:47.137724 21622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:47.147352 21627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:47.147346 21628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:47.147433 21622 server_base.cc:1034] running on GCE node
W20260619 18:47:47.147344 21630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:47.147953 21622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:47.148669 21622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:47.149852 21622 hybrid_clock.cc:648] HybridClock initialized: now 1781894867149829 us; error 37 us; skew 500 ppm
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894867, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:47.152635 21622 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:47:47.153656 21622 webserver.cc:460] Webserver started at http://127.12.234.194:36161/ using document root <none> and password file <none>
I20260619 18:47:47.154241 21622 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:47.154311 21622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:47.154496 21622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:47.156601 21622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "6aa1b867c7a94dd2801bcc5ce33f71ef"
format_stamp: "Formatted at 2026-06-19 18:47:47 on dist-test-slave-fljv"
server_key: "994884791d65f81ce18bc5b844baa0ad"
server_key_iv: "400c4ae2a46fad359e01b04cd7e0e1e7"
server_key_version: "encryptionkey@0"
I20260619 18:47:47.157132 21622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "6aa1b867c7a94dd2801bcc5ce33f71ef"
format_stamp: "Formatted at 2026-06-19 18:47:47 on dist-test-slave-fljv"
server_key: "994884791d65f81ce18bc5b844baa0ad"
server_key_iv: "400c4ae2a46fad359e01b04cd7e0e1e7"
server_key_version: "encryptionkey@0"
I20260619 18:47:47.160921 21622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:47:47.163771 21637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.165124 21622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:47:47.165230 21622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "6aa1b867c7a94dd2801bcc5ce33f71ef"
format_stamp: "Formatted at 2026-06-19 18:47:47 on dist-test-slave-fljv"
server_key: "994884791d65f81ce18bc5b844baa0ad"
server_key_iv: "400c4ae2a46fad359e01b04cd7e0e1e7"
server_key_version: "encryptionkey@0"
I20260619 18:47:47.165345 21622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:47.200307 21622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:47.203534 21622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:47.203725 21622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:47.204381 21622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:47.205330 21622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:47.205391 21622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.205446 21622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:47.205476 21622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.214257 21622 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:42077
I20260619 18:47:47.214313 21750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:42077 every 8 connection(s)
I20260619 18:47:47.215162 21622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:47.224552 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 21622
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894867, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:47.224685 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:47:47.224982 13227 external_mini_cluster.cc:1428] Setting key b362ae53374fd236cba1ef926e908a87
I20260619 18:47:47.229688 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:47.216888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:47829 (local address 127.12.234.254:40337)
0619 18:47:47.217023 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:47.217027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:47.217878 (+ 851us) server_negotiation.cc:407] Connection header received
0619 18:47:47.219365 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:47.219368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:47.219414 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:47.219486 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:47.220788 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.221303 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.222533 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.222769 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.222847 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.225211 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:47.225215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:47.225217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:47.225245 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:47.227365 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:47.227877 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:47.227881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:47.227883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:47.227985 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:47.228402 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:47.228406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:47.228408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:47.228643 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:47.228718 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:47.229371 (+ 653us) server_negotiation.cc:299] Negotiation successful
0619 18:47:47.229513 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:47:47.230445 21753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:47.217303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:47829)
0619 18:47:47.217729 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:47.217772 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:47.219145 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:47.219698 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:47.219705 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:47.220110 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:47.220604 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.220619 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.221422 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.221425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.222440 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.222450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.222896 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.222899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.223115 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.223608 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:47.223623 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:47.225034 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:47.227479 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:47.227483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:47.227496 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:47.227780 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:47.228075 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:47.228078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:47.228079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:47.228285 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:47.228805 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:47.228812 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:47.229114 (+ 302us) client_negotiation.cc:769] Sending connection context
0619 18:47:47.229284 (+ 170us) client_negotiation.cc:240] Negotiation successful
0619 18:47:47.229542 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":101,"threads_started":1}
I20260619 18:47:47.231570 21751 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40337
I20260619 18:47:47.231856 21751 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:47.232362 21751 heartbeater.cc:511] Master 127.12.234.254:40337 requested a full tablet report, sending...
I20260619 18:47:47.233405 21426 ts_manager.cc:194] Registered new tserver with Master: 6aa1b867c7a94dd2801bcc5ce33f71ef (127.12.234.194:42077)
I20260619 18:47:47.233927 21426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:47829
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:47:47.251345 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40337
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:47.359988 21756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:47.360388 21756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:47.360509 21756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:47.364795 21756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:47.364876 21756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:47.364967 21756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:47.370673 21756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:47.371780 21756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:47.372872 21756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:47.382674 21761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:47.382645 21764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:47.382674 21762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:47.382874 21756 server_base.cc:1034] running on GCE node
I20260619 18:47:47.383273 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:47.383886 21756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:47.385046 21756 hybrid_clock.cc:648] HybridClock initialized: now 1781894867385014 us; error 43 us; skew 500 ppm
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894867, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:47.387811 21756 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:47:47.388701 21756 webserver.cc:460] Webserver started at http://127.12.234.195:43699/ using document root <none> and password file <none>
I20260619 18:47:47.389317 21756 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:47.389393 21756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:47.389624 21756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:47:47.391805 21756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "19cd626914c34a588ec19a18b838ad51"
format_stamp: "Formatted at 2026-06-19 18:47:47 on dist-test-slave-fljv"
server_key: "f07f1adf174847989712e9aab2403267"
server_key_iv: "c37def7e2170d28bca01ffb275ac11a4"
server_key_version: "encryptionkey@0"
I20260619 18:47:47.392346 21756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "19cd626914c34a588ec19a18b838ad51"
format_stamp: "Formatted at 2026-06-19 18:47:47 on dist-test-slave-fljv"
server_key: "f07f1adf174847989712e9aab2403267"
server_key_iv: "c37def7e2170d28bca01ffb275ac11a4"
server_key_version: "encryptionkey@0"
I20260619 18:47:47.396229 21756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260619 18:47:47.398787 21771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.400098 21756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:47.400220 21756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "19cd626914c34a588ec19a18b838ad51"
format_stamp: "Formatted at 2026-06-19 18:47:47 on dist-test-slave-fljv"
server_key: "f07f1adf174847989712e9aab2403267"
server_key_iv: "c37def7e2170d28bca01ffb275ac11a4"
server_key_version: "encryptionkey@0"
I20260619 18:47:47.400331 21756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:47.413599 21756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:47.416647 21756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:47.416836 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:47.417447 21756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:47.418334 21756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:47.418380 21756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.418431 21756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:47.418459 21756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.426628 21756 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:38735
I20260619 18:47:47.426651 21884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:38735 every 8 connection(s)
I20260619 18:47:47.427542 21756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:47:47.437310 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 21756
I20260619 18:47:47.437422 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:47:47.437676 13227 external_mini_cluster.cc:1428] Setting key da5530f53d626db2bd38c380986a184d
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894867, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:47:47.443634 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:47.429333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:55861 (local address 127.12.234.254:40337)
0619 18:47:47.429467 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:47.429470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:47.430303 (+ 833us) server_negotiation.cc:407] Connection header received
0619 18:47:47.431547 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:47.431549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:47.431596 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:47.431668 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:47.432860 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.433462 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.434655 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.434883 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.434954 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.440366 (+ 5412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:47.440370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:47.440372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:47.440396 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:47.441577 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:47.442237 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:47.442242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:47.442244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:47.442325 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:47.442679 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:47.442681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:47.442682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:47.442858 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:47.442926 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:47.443405 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 18:47:47.443514 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:47:47.444491 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:47.429691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:55861)
0619 18:47:47.430132 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:47.430206 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:47.431359 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:47.431802 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:47.431809 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:47.432163 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:47.432721 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.432738 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.433586 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.433589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.434560 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.434570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.435000 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.435003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.435173 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.438487 (+ 3314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:47.438510 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:47.440206 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:47:47.441744 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:47.441750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:47.441765 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:47.442027 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:47.442413 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:47:47.442416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:47:47.442418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:47:47.442602 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:47:47.442927 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:47:47.442932 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:47:47.443197 (+ 265us) client_negotiation.cc:769] Sending connection context
0619 18:47:47.443354 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 18:47:47.443653 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
I20260619 18:47:47.445657 21885 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40337
I20260619 18:47:47.445889 21885 heartbeater.cc:464] Registering TS with master...
I20260619 18:47:47.446405 21885 heartbeater.cc:511] Master 127.12.234.254:40337 requested a full tablet report, sending...
I20260619 18:47:47.447458 21426 ts_manager.cc:194] Registered new tserver with Master: 19cd626914c34a588ec19a18b838ad51 (127.12.234.195:38735)
I20260619 18:47:47.447996 21426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:55861
I20260619 18:47:47.451673 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab.
Entry for principal oryx/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab.
WARNING: no policy specified for oryx/127.12.234.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab.
Entry for principal oryx/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab.
WARNING: no policy specified for oryx/127.12.234.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab.
Entry for principal oryx/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab.
I20260619 18:47:47.518546 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 21488
I20260619 18:47:47.523408 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 21622
I20260619 18:47:47.527977 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 21756
I20260619 18:47:47.532519 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 21394
I20260619 18:47:47.537492 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40337
--webserver_interface=127.12.234.254
--webserver_port=35477
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:40337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:47:47.643980 21894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:47.644269 21894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:47.644325 21894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:47.648391 21894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:47:47.648453 21894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:47.648479 21894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:47.648500 21894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:47:47.648524 21894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:47:47.653856 21894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:40337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:47.654969 21894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:47.655970 21894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:47.664176 21900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:47.664192 21902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:47.664186 21899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:47.664508 21894 server_base.cc:1034] running on GCE node
I20260619 18:47:47.665081 21894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:47.666145 21894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:47.667335 21894 hybrid_clock.cc:648] HybridClock initialized: now 1781894867667304 us; error 42 us; skew 500 ppm
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894867, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:47.669955 21894 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:47:47.670845 21894 webserver.cc:460] Webserver started at http://127.12.234.254:35477/ using document root <none> and password file <none>
I20260619 18:47:47.671367 21894 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:47.671445 21894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:47.675254 21894 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260619 18:47:47.677313 21909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.678553 21894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:47.678658 21894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "b0f9c0f343b14502b6e3c89c499f6f21"
format_stamp: "Formatted at 2026-06-19 18:47:46 on dist-test-slave-fljv"
server_key: "dfc66debe7d852be4257de9b1564d36a"
server_key_iv: "dfe741b75084477fc8872e493edf56cc"
server_key_version: "encryptionkey@0"
I20260619 18:47:47.679200 21894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:47.695214 21894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:47.698103 21894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:47.698311 21894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:47.705185 21894 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:40337
I20260619 18:47:47.705233 21961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:40337 every 8 connection(s)
I20260619 18:47:47.706287 21894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:47:47.710645 21962 sys_catalog.cc:263] Verifying existing consensus state
I20260619 18:47:47.712342 21962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: Bootstrap starting.
I20260619 18:47:47.712951 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 21894
I20260619 18:47:47.713424 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:42071
--local_ip_for_outbound_sockets=127.12.234.193
--tserver_master_addrs=127.12.234.254:40337
--webserver_port=45347
--webserver_interface=127.12.234.193
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.234.193
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:47:47.728888 21962 log.cc:824] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: Log is configured to *not* fsync() on all Append() calls
I20260619 18:47:47.735466 21962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:47:47.735802 21962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: Bootstrap complete.
I20260619 18:47:47.739850 21962 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } }
I20260619 18:47:47.740324 21962 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f9c0f343b14502b6e3c89c499f6f21, State: Initialized, Role: FOLLOWER
I20260619 18:47:47.740832 21962 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } }
I20260619 18:47:47.740962 21962 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:47:47.741029 21962 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:47:47.741118 21962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 1 FOLLOWER]: Advancing to term 2
I20260619 18:47:47.743147 21962 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } }
I20260619 18:47:47.743461 21962 leader_election.cc:302] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0f9c0f343b14502b6e3c89c499f6f21; no voters:
I20260619 18:47:47.743783 21962 leader_election.cc:288] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 18:47:47.743863 21966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 18:47:47.744037 21966 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [term 2 LEADER]: Becoming Leader. State: Replica: b0f9c0f343b14502b6e3c89c499f6f21, State: Running, Role: LEADER
I20260619 18:47:47.744340 21966 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } }
I20260619 18:47:47.745042 21962 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:47:47.745335 21968 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0f9c0f343b14502b6e3c89c499f6f21. Latest consensus state: current_term: 2 leader_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } } }
I20260619 18:47:47.745465 21968 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:47.745702 21967 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f9c0f343b14502b6e3c89c499f6f21" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40337 } } }
I20260619 18:47:47.745801 21967 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21 [sys.catalog]: This master's current role is: LEADER
I20260619 18:47:47.746127 21970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:47:47.749491 21970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:47:47.750099 21970 catalog_manager.cc:1259] Loaded cluster ID: 3177310961db4ef3bde7adfb0b14690b
I20260619 18:47:47.750147 21970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:47:47.751775 21970 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:47:47.752291 21970 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b0f9c0f343b14502b6e3c89c499f6f21: Loaded TSK: 0
I20260619 18:47:47.753183 21970 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 18:47:47.838207 21964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:47.838487 21964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:47.838557 21964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:47.842702 21964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:47.842773 21964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:47.842852 21964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:47:47.848362 21964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.193.keytab
--principal=oryx/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:42071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=45347
--tserver_master_addrs=127.12.234.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:47.849444 21964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:47.850507 21964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:47.859930 21988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:47.859930 21989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:47.860041 21991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:47.860371 21964 server_base.cc:1034] running on GCE node
I20260619 18:47:47.860766 21964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:47.861438 21964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:47.862624 21964 hybrid_clock.cc:648] HybridClock initialized: now 1781894867862597 us; error 39 us; skew 500 ppm
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894867, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:47.865329 21964 init.cc:377] Logged in from keytab as oryx/127.12.234.193@KRBTEST.COM (short username oryx)
I20260619 18:47:47.866248 21964 webserver.cc:460] Webserver started at http://127.12.234.193:45347/ using document root <none> and password file <none>
I20260619 18:47:47.866814 21964 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:47.866884 21964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:47.870715 21964 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:47:47.872816 21998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.874253 21964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:47:47.874361 21964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "96214954997d4016b67f15731bbbd6d9"
format_stamp: "Formatted at 2026-06-19 18:47:46 on dist-test-slave-fljv"
server_key: "8f1e28302f987fddddaac129ad2eacda"
server_key_iv: "41b6f06246a85e2c30909bd2600eb6cb"
server_key_version: "encryptionkey@0"
I20260619 18:47:47.874887 21964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:47.887709 21964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:47.890831 21964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:47.891010 21964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:47.891578 21964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:47.892374 21964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:47.892421 21964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.892477 21964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:47.892500 21964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:47.901288 21964 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:42071
I20260619 18:47:47.901317 22111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:42071 every 8 connection(s)
I20260619 18:47:47.902346 21964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:47:47.909966 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 21964
I20260619 18:47:47.910492 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:42077
--local_ip_for_outbound_sockets=127.12.234.194
--tserver_master_addrs=127.12.234.254:40337
--webserver_port=36161
--webserver_interface=127.12.234.194
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.234.194
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:47.915078 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:47.904469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:50561)
0619 18:47:47.904897 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:47.904938 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:47.906055 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:47.907126 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:47.907133 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:47.907480 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:47.907945 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.907958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.909202 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.909205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.910370 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.910382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.911221 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.911225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.911411 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.912048 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:47.912064 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:47.914315 (+ 2251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":134,"threads_started":1}
W20260619 18:47:47.915535 22114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:47.904370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:50561 (local address 127.12.234.254:40337)
0619 18:47:47.904816 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:47.904835 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:47.905043 (+ 208us) server_negotiation.cc:407] Connection header received
0619 18:47:47.906449 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:47.906461 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:47.906771 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:47.907043 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:47.908079 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.909068 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.910483 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.911056 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.911217 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.914606 (+ 3389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:50561: BlockingRecv error: recv got EOF from 127.12.234.193:50561 (error 108)
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":91,"threads_started":1}
W20260619 18:47:47.915625 22112 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:40337 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.234.254:40337: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:47.921082 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:47.916037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:58329)
0619 18:47:47.916177 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:47.916193 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:47.916276 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:47.916494 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:47.916497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:47.916660 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:47.916777 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.916784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.917466 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.917469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.918381 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.918392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.918800 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.918803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.918897 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.919329 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:47.919343 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:47.920989 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 18:47:47.921257 22114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:47.916179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:58329 (local address 127.12.234.254:40337)
0619 18:47:47.916284 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:47.916287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:47.916299 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:47:47.916359 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:47.916361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:47.916398 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:47.916461 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:47.916874 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.917374 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.918484 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.918685 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.918773 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.921154 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:58329: BlockingRecv error: recv got EOF from 127.12.234.193:58329 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:47 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:47.927572 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:47.921571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:55267)
0619 18:47:47.921869 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:47.921883 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:47.921974 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:47.922199 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:47.922202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:47.922395 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:47.922500 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.922506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.923134 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.923136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.924036 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:47.924045 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.924647 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.924651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:47.924797 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.925287 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:47.925300 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:47.927414 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
W20260619 18:47:47.927793 22114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:47.921632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:55267 (local address 127.12.234.254:40337)
0619 18:47:47.921746 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:47.921750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:47.921949 (+ 199us) server_negotiation.cc:407] Connection header received
0619 18:47:47.922049 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:47.922051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:47.922087 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:47.922147 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:47.922588 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.923047 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.924170 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:47.924522 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:47.924624 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:47.927624 (+ 3000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:55267: BlockingRecv error: recv got EOF from 127.12.234.193:55267 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 18:47:47.927917 22112 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 18:47:48.023133 22116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:48.023422 22116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:48.023481 22116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:48.027541 22116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:48.027616 22116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:48.027715 22116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:47:48.033123 22116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.194.keytab
--principal=oryx/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:42077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=36161
--tserver_master_addrs=127.12.234.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:48.034303 22116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:48.035375 22116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:48.044993 22123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:48.044993 22122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:48.045019 22125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:48.045051 22116 server_base.cc:1034] running on GCE node
I20260619 18:47:48.045662 22116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:48.046437 22116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:48.047593 22116 hybrid_clock.cc:648] HybridClock initialized: now 1781894868047569 us; error 34 us; skew 500 ppm
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894868, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:48.050269 22116 init.cc:377] Logged in from keytab as oryx/127.12.234.194@KRBTEST.COM (short username oryx)
I20260619 18:47:48.051095 22116 webserver.cc:460] Webserver started at http://127.12.234.194:36161/ using document root <none> and password file <none>
I20260619 18:47:48.051612 22116 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:48.051679 22116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:48.055573 22116 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:47:48.057713 22132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:48.058987 22116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:47:48.059103 22116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "6aa1b867c7a94dd2801bcc5ce33f71ef"
format_stamp: "Formatted at 2026-06-19 18:47:47 on dist-test-slave-fljv"
server_key: "994884791d65f81ce18bc5b844baa0ad"
server_key_iv: "400c4ae2a46fad359e01b04cd7e0e1e7"
server_key_version: "encryptionkey@0"
I20260619 18:47:48.059661 22116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:48.085356 22116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:48.088667 22116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:48.088850 22116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:48.089448 22116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:48.090327 22116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:48.090374 22116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:48.090428 22116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:48.090456 22116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:48.099164 22116 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:42077
I20260619 18:47:48.099301 22245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:42077 every 8 connection(s)
I20260619 18:47:48.100198 22116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:47:48.106876 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 22116
I20260619 18:47:48.107349 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:38735
--local_ip_for_outbound_sockets=127.12.234.195
--tserver_master_addrs=127.12.234.254:40337
--webserver_port=43699
--webserver_interface=127.12.234.195
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.234.195
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 18:47:48.111722 22114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:48.102126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:53785 (local address 127.12.234.254:40337)
0619 18:47:48.102295 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:48.102299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:48.103126 (+ 827us) server_negotiation.cc:407] Connection header received
0619 18:47:48.104358 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:48.104361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:48.104414 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:48.104493 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:48.105627 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.106161 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.107650 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.107862 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.107939 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.111577 (+ 3638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:53785: BlockingRecv error: recv got EOF from 127.12.234.194:53785 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 18:47:48.112561 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:48.102501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:53785)
0619 18:47:48.102973 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:48.103017 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:48.104142 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:48.104652 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:48.104658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:48.105037 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:48.105515 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.105528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.106298 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.106301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.107538 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.107548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.108000 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.108004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.108196 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.108795 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:48.108810 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:48.111398 (+ 2588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
W20260619 18:47:48.113101 22246 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:40337 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.234.254:40337: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:48.118369 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:48.113533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:35847)
0619 18:47:48.113759 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:48.113772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:48.113855 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:48.114067 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:48.114070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:48.114278 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:48.114367 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.114372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.115054 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.115058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.115601 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.115609 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.115998 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.116001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.116085 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.116587 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:48.116599 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:48.118249 (+ 1650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168}
W20260619 18:47:48.118475 22114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:48.113651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:35847 (local address 127.12.234.254:40337)
0619 18:47:48.113754 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:48.113758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:48.113830 (+ 72us) server_negotiation.cc:407] Connection header received
0619 18:47:48.113941 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:48.113943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:48.113982 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:48.114043 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:48.114466 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.114962 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.115704 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.115907 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.115988 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.118369 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:35847: BlockingRecv error: recv got EOF from 127.12.234.194:35847 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:48.123625 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:48.118852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:34639)
0619 18:47:48.118953 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:48.118966 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:48.119038 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:48.119242 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:48.119245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:48.119396 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:48.119482 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.119488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.120116 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.120119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.120977 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.120985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.121352 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.121355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.121435 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.121854 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:48.121867 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:48.123503 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 18:47:48.123733 22114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:48.118947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:34639 (local address 127.12.234.254:40337)
0619 18:47:48.119037 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:48.119040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:48.119052 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:47:48.119118 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:48.119120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:48.119156 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:48.119213 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:48.119566 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.120040 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.121071 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.121269 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.121335 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.123635 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:34639: BlockingRecv error: recv got EOF from 127.12.234.194:34639 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 18:47:48.123958 22246 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 18:47:48.219976 22249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:47:48.220299 22249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:47:48.220362 22249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:47:48.225070 22249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:47:48.225162 22249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:47:48.225261 22249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:47:48.232596 22249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/krb5kdc/oryx_127.12.234.195.keytab
--principal=oryx/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:38735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=43699
--tserver_master_addrs=127.12.234.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:47:48.233738 22249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:47:48.234818 22249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:47:48.244702 22257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:48.244700 22249 server_base.cc:1034] running on GCE node
W20260619 18:47:48.244764 22255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:47:48.244730 22254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:47:48.245399 22249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:47:48.246124 22249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:47:48.247311 22249 hybrid_clock.cc:648] HybridClock initialized: now 1781894868247287 us; error 40 us; skew 500 ppm
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894868, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:47:48.250042 22249 init.cc:377] Logged in from keytab as oryx/127.12.234.195@KRBTEST.COM (short username oryx)
I20260619 18:47:48.251032 22249 webserver.cc:460] Webserver started at http://127.12.234.195:43699/ using document root <none> and password file <none>
I20260619 18:47:48.251605 22249 fs_manager.cc:359] Metadata directory not provided
I20260619 18:47:48.251688 22249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:47:48.255712 22249 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:47:48.257918 22264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:48.259254 22249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:47:48.259378 22249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "19cd626914c34a588ec19a18b838ad51"
format_stamp: "Formatted at 2026-06-19 18:47:47 on dist-test-slave-fljv"
server_key: "f07f1adf174847989712e9aab2403267"
server_key_iv: "c37def7e2170d28bca01ffb275ac11a4"
server_key_version: "encryptionkey@0"
I20260619 18:47:48.259943 22249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:47:48.279927 22249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:47:48.283383 22249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:47:48.283563 22249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:47:48.284143 22249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:47:48.285012 22249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:47:48.285059 22249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:48.285113 22249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:47:48.285138 22249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:47:48.294073 22249 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:38735
I20260619 18:47:48.294100 22377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:38735 every 8 connection(s)
I20260619 18:47:48.295163 22249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:48.304993 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 22249
W20260619 18:47:48.305823 22114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:48.296928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:36829 (local address 127.12.234.254:40337)
0619 18:47:48.297062 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:48.297065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:48.297961 (+ 896us) server_negotiation.cc:407] Connection header received
0619 18:47:48.299257 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:48.299260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:48.299316 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:48.299390 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:48.300537 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.301057 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.302165 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.302439 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.302530 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.305701 (+ 3171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:36829: BlockingRecv error: recv got EOF from 127.12.234.195:36829 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 18:47:48.306525 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:48.297389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:36829)
0619 18:47:48.297818 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:48.297860 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:48.299049 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:48.299558 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:48.299565 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:48.299915 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:48.300397 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.300410 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.301175 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.301177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.302049 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.302058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.302580 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.302584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.302755 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.303302 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:48.303317 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:48.305562 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":101,"threads_started":1}
W20260619 18:47:48.307090 22378 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:40337 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.234.254:40337: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:48.312947 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:48.307512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:53083)
0619 18:47:48.307665 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:48.307677 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:48.307769 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:48.307961 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:48.307964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:48.308116 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:48.308215 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.308221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.309196 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.309200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.309927 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.309937 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.310394 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.310397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.310492 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.310894 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:48.310904 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:48.312812 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
W20260619 18:47:48.313056 22382 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:48.307572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:53083 (local address 127.12.234.254:40337)
0619 18:47:48.307782 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:48.307785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:48.307813 (+ 28us) server_negotiation.cc:407] Connection header received
0619 18:47:48.307840 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:48.307842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:48.307875 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:48.307935 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:48.308307 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.309076 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.310031 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.310290 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.310369 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.312943 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:53083: BlockingRecv error: recv got EOF from 127.12.234.195:53083 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":78,"threads_started":1}
I20260619 18:47:48.315578 22114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:48.305979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33124 (local address 127.12.234.254:40337)
0619 18:47:48.306078 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:48.306082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:48.306276 (+ 194us) server_negotiation.cc:407] Connection header received
0619 18:47:48.306403 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:48.306407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:48.306448 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:48.306512 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:48.307345 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.308027 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.309196 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.309538 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.309642 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.311179 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:47:48.311186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:47:48.311219 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:47:48.311253 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:47:48.313124 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:48.313768 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:48.313776 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:48.313783 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:48.313928 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:47:48.314314 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:47:48.314317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:47:48.314319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:47:48.314687 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:47:48.314838 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 18:47:48.315166 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 18:47:48.315428 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:48.318537 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:48.313553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:57875)
0619 18:47:48.313661 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:48.313675 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:48.313752 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:48.313976 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:48.313979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:48.314116 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:48.314239 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.314246 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.315012 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.315015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.315755 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.315766 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.316156 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.316159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.316230 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.316648 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:48.316660 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:48.318417 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 18:47:48.318632 22382 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:48.313671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:57875 (local address 127.12.234.254:40337)
0619 18:47:48.313811 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:48.313814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:48.313823 (+ 9us) server_negotiation.cc:407] Connection header received
0619 18:47:48.313851 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:48.313853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:48.313885 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:48.313941 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:48.314346 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.314884 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.315857 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.316066 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.316145 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.318542 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:57875: BlockingRecv error: recv got EOF from 127.12.234.195:57875 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260619 18:47:48.318796 22378 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:48 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:48.933797 22383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:48.928532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:50703)
0619 18:47:48.928784 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:48.928801 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:48.928909 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:48.929144 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:48.929146 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:48.929312 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:48.929428 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.929434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.930224 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.930228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.930916 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:48.930926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.931402 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.931406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:48.931500 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.931925 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:48.931935 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:48.933677 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":68,"threads_started":1}
W20260619 18:47:48.933902 22384 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:48.928673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:50703 (local address 127.12.234.254:40337)
0619 18:47:48.928933 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:48.928936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:48.928964 (+ 28us) server_negotiation.cc:407] Connection header received
0619 18:47:48.928999 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:48.929002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:48.929042 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:48.929109 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:48.929527 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.930076 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.931023 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:48.931285 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:48.931385 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:48.933792 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:50703: BlockingRecv error: recv got EOF from 127.12.234.193:50703 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":102,"threads_started":1}
Jun 19 18:47:49 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:49 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:49.129654 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:49.124513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:34577)
0619 18:47:49.124761 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:49.124774 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:49.124869 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:49.125075 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:49.125077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:49.125251 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:49.125349 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:49.125354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.126030 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.126032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:49.126667 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:49.126677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.127090 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.127092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:49.127160 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:49.127637 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:49.127649 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:49.129511 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":69,"threads_started":1}
W20260619 18:47:49.129753 22384 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:49.124582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:34577 (local address 127.12.234.254:40337)
0619 18:47:49.124706 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:49.124709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:49.124834 (+ 125us) server_negotiation.cc:407] Connection header received
0619 18:47:49.124941 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:49.124943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:49.124984 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:49.125050 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:49.125435 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.125945 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.126780 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.127000 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.127078 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:49.129656 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:34577: BlockingRecv error: recv got EOF from 127.12.234.194:34577 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 19 18:47:49 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:49 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:49.324663 22386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:49.319464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:52717)
0619 18:47:49.319723 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:49.319738 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:49.319847 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:49.320068 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:49.320070 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:49.320254 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:49.320349 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:49.320355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.321061 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.321063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:49.321636 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:49.321645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.322024 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.322026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:49.322083 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:49.322593 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:49.322603 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:49.324506 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":71,"threads_started":1}
W20260619 18:47:49.324761 22384 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:49.319555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:52717 (local address 127.12.234.254:40337)
0619 18:47:49.319692 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:49.319697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:49.319808 (+ 111us) server_negotiation.cc:407] Connection header received
0619 18:47:49.319923 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:49.319926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:49.319972 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:49.320040 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:49.320445 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.320969 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.321727 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.321939 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.322010 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:49.324651 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:52717: BlockingRecv error: recv got EOF from 127.12.234.195:52717 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 18:47:49 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:49 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:49.939679 22387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:49.934502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:55277)
0619 18:47:49.934752 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:49.934767 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:49.934857 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:49.935073 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:49.935075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:49.935262 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:49.935358 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:49.935364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.936052 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.936054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:49.936609 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:49.936616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.937014 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.937017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:49.937079 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:49.937670 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:49.937682 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:49.939537 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":71,"threads_started":1}
W20260619 18:47:49.939812 22388 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:49.934565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:55277 (local address 127.12.234.254:40337)
0619 18:47:49.934797 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:49.934800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:49.934813 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:47:49.934925 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:49.934927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:49.934971 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:49.935052 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:49.935449 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.935974 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.936697 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:49.936911 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:49.936982 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:49.939696 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:55277: BlockingRecv error: recv got EOF from 127.12.234.193:55277 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":83,"threads_started":1}
Jun 19 18:47:50 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:50 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:50.136250 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:50.130542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:53271)
0619 18:47:50.130830 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:50.130844 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:50.130956 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:50.131319 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:50.131322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:50.131586 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:50.131697 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:50.131704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.132469 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.132472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:50.133045 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:50.133053 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.133515 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.133519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:50.133610 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:50.134125 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:50.134137 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:50.136097 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":98,"threads_started":1}
W20260619 18:47:50.136338 22388 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:50.130630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:53271 (local address 127.12.234.254:40337)
0619 18:47:50.130780 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:50.130786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:50.130934 (+ 148us) server_negotiation.cc:407] Connection header received
0619 18:47:50.131126 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:50.131130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:50.131189 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:50.131277 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:50.131820 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.132358 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.133162 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.133395 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.133484 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:50.136240 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:53271: BlockingRecv error: recv got EOF from 127.12.234.194:53271 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 18:47:50 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:50 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:50.331152 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:50.325678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:48773)
0619 18:47:50.325977 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:50.325991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:50.326100 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:50.326369 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:50.326371 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:50.326565 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:50.326666 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:50.326672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.327449 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.327452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:50.328054 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:50.328063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.328489 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.328493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:50.328564 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:50.329079 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:50.329091 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:50.330985 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
W20260619 18:47:50.331434 22388 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:50.325864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:48773 (local address 127.12.234.254:40337)
0619 18:47:50.325991 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:50.325995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:50.326050 (+ 55us) server_negotiation.cc:407] Connection header received
0619 18:47:50.326204 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:50.326206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:50.326267 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:50.326342 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:50.326788 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.327327 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.328147 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.328370 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.328441 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:50.331208 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:48773: BlockingRecv error: recv got EOF from 127.12.234.195:48773 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 18:47:50 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:50 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:50.946058 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:50.940660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:57487)
0619 18:47:50.940909 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:50.940923 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:50.941021 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:50.941328 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:50.941330 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:50.941509 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:50.941608 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:50.941614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.942369 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.942371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:50.942919 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:50.942926 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.943438 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.943441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:50.943514 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:50.943966 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:50.943977 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:50.945927 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":83,"threads_started":1}
W20260619 18:47:50.946329 22392 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:50.940840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:57487 (local address 127.12.234.254:40337)
0619 18:47:50.941064 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:50.941068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:50.941081 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:47:50.941121 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:50.941123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:50.941164 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:50.941254 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:50.941718 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.942289 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.943106 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:50.943334 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:50.943432 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:50.946155 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:57487: BlockingRecv error: recv got EOF from 127.12.234.193:57487 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
Jun 19 18:47:51 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:51 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:51.142536 22393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:51.137173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:48461)
0619 18:47:51.137432 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:51.137446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:51.137565 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:51.137855 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:51.137858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:51.138066 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:51.138230 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:51.138240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.139060 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.139064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:51.139649 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:51.139657 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.140049 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.140051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:51.140110 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:51.140548 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:51.140558 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:51.142415 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":69,"threads_started":1}
W20260619 18:47:51.142661 22392 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:51.137315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:48461 (local address 127.12.234.254:40337)
0619 18:47:51.137495 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:51.137499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:51.137514 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:51.137679 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:51.137682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:51.137728 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:51.137800 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:51.138349 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.138946 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.139752 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.139967 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.140042 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:51.142555 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:48461: BlockingRecv error: recv got EOF from 127.12.234.194:48461 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Jun 19 18:47:51 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:51 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:51.337294 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:51.331935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:45479)
0619 18:47:51.332251 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:51.332266 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:51.332378 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:51.332617 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:51.332620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:51.332806 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:51.332894 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:51.332899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.333730 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.333734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:51.334355 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:51.334364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.334805 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.334807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:51.334865 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:51.335354 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:51.335365 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:51.337175 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":104,"threads_started":1}
W20260619 18:47:51.337492 22392 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:51.332045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:45479 (local address 127.12.234.254:40337)
0619 18:47:51.332215 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:51.332220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:51.332333 (+ 113us) server_negotiation.cc:407] Connection header received
0619 18:47:51.332462 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:51.332465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:51.332510 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:51.332575 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:51.333001 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.333580 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.334474 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.334698 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.334788 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:51.337358 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:45479: BlockingRecv error: recv got EOF from 127.12.234.195:45479 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Jun 19 18:47:51 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:51 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:51.951849 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:51.946832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:58409)
0619 18:47:51.947157 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:51.947171 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:51.947273 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:51.947492 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:51.947494 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:51.947680 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:51.947775 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:51.947781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.948475 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.948477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:51.949078 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:51.949087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.949478 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.949480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:51.949538 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:51.950005 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:51.950016 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:51.951734 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":65,"threads_started":1}
W20260619 18:47:51.951995 22396 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:51.946889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:58409 (local address 127.12.234.254:40337)
0619 18:47:51.947152 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:51.947156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:51.947223 (+ 67us) server_negotiation.cc:407] Connection header received
0619 18:47:51.947345 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:51.947347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:51.947392 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:51.947466 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:51.947861 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.948372 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.949172 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:51.949399 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:51.949470 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:51.951872 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:58409: BlockingRecv error: recv got EOF from 127.12.234.193:58409 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
Jun 19 18:47:52 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:52 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:52.148633 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:52.143281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:42177)
0619 18:47:52.143580 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:52.143595 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:52.143694 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:52.143982 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:52.143984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:52.144153 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:52.144254 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:52.144259 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.145057 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.145061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:52.145781 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:52.145792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.146248 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.146252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:52.146337 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:52.146811 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:52.146822 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:52.148508 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":71,"threads_started":1}
W20260619 18:47:52.148766 22396 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:52.143383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:42177 (local address 127.12.234.254:40337)
0619 18:47:52.143530 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:52.143533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:52.143660 (+ 127us) server_negotiation.cc:407] Connection header received
0619 18:47:52.143814 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:52.143817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:52.143870 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:52.143938 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:52.144356 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.144886 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.145896 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.146124 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.146235 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:52.148646 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:42177: BlockingRecv error: recv got EOF from 127.12.234.194:42177 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 18:47:52 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:52 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:52.343439 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:52.338114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:52451)
0619 18:47:52.338440 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:52.338453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:52.338555 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:52.338809 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:52.338811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:52.338983 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:52.339077 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:52.339082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.339858 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.339860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:52.340410 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:52.340417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.340901 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.340904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:52.340968 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:52.341426 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:52.341438 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:52.343317 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":139,"threads_started":1}
W20260619 18:47:52.343716 22396 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:52.338282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:52451 (local address 127.12.234.254:40337)
0619 18:47:52.338450 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:52.338454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:52.338516 (+ 62us) server_negotiation.cc:407] Connection header received
0619 18:47:52.338643 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:52.338645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:52.338706 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:52.338766 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:52.339202 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.339760 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.340563 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.340792 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.340878 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:52.343482 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:52451: BlockingRecv error: recv got EOF from 127.12.234.195:52451 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Jun 19 18:47:52 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:52 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:52.963397 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:52.952556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:40763)
0619 18:47:52.952827 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:52.952843 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:52.952930 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:52.953169 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:52.953172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:52.953346 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:52.953451 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:52.953457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.954155 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.954158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:52.954756 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:52.954763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.955157 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.955161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:52.955228 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:52.961291 (+ 6063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:52.961304 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:52.963259 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":73,"threads_started":1}
W20260619 18:47:52.963537 22400 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:52.952653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:40763 (local address 127.12.234.254:40337)
0619 18:47:52.952914 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:52.952917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:52.952931 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:52.953004 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:52.953006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:52.953046 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:52.953121 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:52.953547 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.954068 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.954841 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:52.955062 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:52.955133 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:52.963403 (+ 8270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:40763: BlockingRecv error: recv got EOF from 127.12.234.193:40763 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
Jun 19 18:47:53 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:53 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:53.154369 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:53.149325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:46673)
0619 18:47:53.149591 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:53.149605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:53.149714 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:53.149929 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:53.149932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:53.150131 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:53.150259 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:53.150265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.150974 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.150978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:53.151540 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:53.151548 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.151957 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.151960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:53.152038 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:53.152490 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:53.152500 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:53.154253 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":76,"threads_started":1}
W20260619 18:47:53.154472 22400 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:53.149455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:46673 (local address 127.12.234.254:40337)
0619 18:47:53.149592 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:53.149596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:53.149683 (+ 87us) server_negotiation.cc:407] Connection header received
0619 18:47:53.149798 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:53.149801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:53.149843 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:53.149902 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:53.150367 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.150877 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.151645 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.151863 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.151944 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:53.154377 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:46673: BlockingRecv error: recv got EOF from 127.12.234.194:46673 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 18:47:53 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:53 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:53.349803 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:53.344335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:59343)
0619 18:47:53.344607 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:53.344623 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:53.344764 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:53.345009 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:53.345012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:53.345228 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:53.345370 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:53.345378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.346152 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.346156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:53.346860 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:53.346871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.347287 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.347291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:53.347380 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:53.347886 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:53.347901 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:53.349685 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
W20260619 18:47:53.349948 22400 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:53.344458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:59343 (local address 127.12.234.254:40337)
0619 18:47:53.344597 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:53.344601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:53.344696 (+ 95us) server_negotiation.cc:407] Connection header received
0619 18:47:53.344861 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:53.344863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:53.344909 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:53.344975 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:53.345485 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.346045 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.346971 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.347183 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.347282 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:53.349838 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:59343: BlockingRecv error: recv got EOF from 127.12.234.195:59343 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 19 18:47:53 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:53 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:53.969776 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:53.964198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:34089)
0619 18:47:53.964473 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:53.964486 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:53.964579 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:53.964826 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:53.964828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:53.964996 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:53.965090 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:53.965095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.965806 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.965896 (+ 90us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:53.966636 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:53.966649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.967145 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.967148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:53.967222 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:53.967681 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:53.967692 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:53.969627 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":92,"threads_started":1}
W20260619 18:47:53.969902 22404 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:53.964289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:34089 (local address 127.12.234.254:40337)
0619 18:47:53.964531 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:53.964536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:53.964550 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:47:53.964659 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:53.964662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:53.964705 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:53.964803 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:53.965174 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.965685 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.966790 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:53.967018 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:53.967099 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:53.969793 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:34089: BlockingRecv error: recv got EOF from 127.12.234.193:34089 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":55,"threads_started":1}
Jun 19 18:47:54 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:54 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:54.160966 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:54.155241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:51385)
0619 18:47:54.155537 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:54.155551 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:54.155675 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:54.155946 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:54.155951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:54.156207 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:54.156310 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:54.156316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.157218 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.157221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:54.157853 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:54.157863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.158382 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.158386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:54.158457 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:54.158927 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:54.158938 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:54.160845 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
W20260619 18:47:54.161129 22404 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:54.155339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:51385 (local address 127.12.234.254:40337)
0619 18:47:54.155468 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:54.155472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:54.155624 (+ 152us) server_negotiation.cc:407] Connection header received
0619 18:47:54.155767 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:54.155770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:54.155814 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:54.155894 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:54.156455 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.157075 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.157974 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.158266 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.158357 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:54.161007 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:51385: BlockingRecv error: recv got EOF from 127.12.234.194:51385 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 18:47:54 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:54 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:54.356160 22406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:54.350654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:36093)
0619 18:47:54.350980 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:54.350996 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:54.351123 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:54.351383 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:54.351386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:54.351583 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:54.351676 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:54.351682 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.352469 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.352473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:54.353033 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:54.353041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.353486 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.353490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:54.353567 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:54.354062 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:54.354075 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:54.356005 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":84,"threads_started":1}
W20260619 18:47:54.356268 22404 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:54.350791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:36093 (local address 127.12.234.254:40337)
0619 18:47:54.350958 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:54.350964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:54.351073 (+ 109us) server_negotiation.cc:407] Connection header received
0619 18:47:54.351214 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:54.351217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:54.351274 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:54.351343 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:54.351801 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.352359 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.353151 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.353379 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.353469 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:54.356164 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:36093: BlockingRecv error: recv got EOF from 127.12.234.195:36093 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Jun 19 18:47:54 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:54 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:54.976032 22407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:54.970709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:41345)
0619 18:47:54.971054 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:54.971071 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:54.971196 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:54.971468 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:54.971471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:54.971664 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:54.971785 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:54.971791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.972534 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.972538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:54.973089 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:54.973097 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.973522 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.973525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:54.973603 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:54.974126 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:54.974137 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:54.975884 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":89,"threads_started":1}
W20260619 18:47:54.976135 22408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:54.970798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:41345 (local address 127.12.234.254:40337)
0619 18:47:54.971056 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:54.971061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:54.971152 (+ 91us) server_negotiation.cc:407] Connection header received
0619 18:47:54.971310 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:54.971312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:54.971358 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:54.971435 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:54.971903 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.972435 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.973197 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:54.973427 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:54.973508 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:54.976026 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:41345: BlockingRecv error: recv got EOF from 127.12.234.193:41345 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":73,"threads_started":1}
Jun 19 18:47:55 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:55 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:55.169596 22409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:55.161618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:34607)
0619 18:47:55.161899 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:55.161912 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:55.162007 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:55.162306 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:55.162308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:55.162512 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:55.162614 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:55.162621 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.163483 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.163486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:55.164138 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:55.164148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.164604 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.164606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:55.164665 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:55.167678 (+ 3013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:55.167692 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:55.169476 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":75,"threads_started":1}
W20260619 18:47:55.169754 22408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:55.161684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:34607 (local address 127.12.234.254:40337)
0619 18:47:55.161815 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:55.161819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:55.161969 (+ 150us) server_negotiation.cc:407] Connection header received
0619 18:47:55.162075 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:55.162077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:55.162119 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:55.162289 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:55.162744 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.163380 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.164261 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.164509 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.164596 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:55.169629 (+ 5033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:34607: BlockingRecv error: recv got EOF from 127.12.234.194:34607 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 18:47:55 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:55 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:55.362219 22410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:55.356957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:37411)
0619 18:47:55.357313 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:55.357326 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:55.357430 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:55.357717 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:55.357719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:55.357893 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:55.357990 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:55.357995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.358770 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.358773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:55.359334 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:55.359342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.359759 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.359762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:55.359824 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:55.360286 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:55.360299 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:55.362037 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":133,"threads_started":1}
W20260619 18:47:55.362345 22408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:55.357046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:37411 (local address 127.12.234.254:40337)
0619 18:47:55.357159 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:55.357162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:55.357420 (+ 258us) server_negotiation.cc:407] Connection header received
0619 18:47:55.357503 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:55.357506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:55.357555 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:55.357655 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:55.358092 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.358666 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.359434 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.359665 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.359742 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:55.362211 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:37411: BlockingRecv error: recv got EOF from 127.12.234.195:37411 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 18:47:55 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:55 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:55.982625 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:55.976917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:56117)
0619 18:47:55.977239 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:55.977256 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:55.977366 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:55.977593 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:55.977595 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:55.977784 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:55.977881 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:55.977886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.978749 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.978759 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:55.979427 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:55.979439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.979958 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.979961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:55.980032 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:55.980570 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:55.980584 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:55.982481 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":78,"threads_started":1}
W20260619 18:47:55.982728 22412 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:55.977160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:56117 (local address 127.12.234.254:40337)
0619 18:47:55.977393 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:55.977396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:55.977411 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:55.977450 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:55.977452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:55.977494 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:55.977571 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:55.977996 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.978626 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.979528 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:55.979843 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:55.979932 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:55.982607 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:56117: BlockingRecv error: recv got EOF from 127.12.234.193:56117 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":74,"threads_started":1}
Jun 19 18:47:56 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:56 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:56.175781 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:56.170366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:60985)
0619 18:47:56.170646 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:56.170658 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:56.170755 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:56.170970 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:56.170972 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:56.171147 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:56.171242 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:56.171247 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.172226 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.172230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:56.172809 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:56.172818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.173218 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.173221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:56.173287 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:56.173743 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:56.173753 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:56.175625 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":81,"threads_started":1}
W20260619 18:47:56.175891 22412 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:56.170452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:60985 (local address 127.12.234.254:40337)
0619 18:47:56.170590 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:56.170594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:56.170726 (+ 132us) server_negotiation.cc:407] Connection header received
0619 18:47:56.170829 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:56.170831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:56.170876 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:56.170941 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:56.171378 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.172073 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.172912 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.173123 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.173194 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:56.175789 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:60985: BlockingRecv error: recv got EOF from 127.12.234.194:60985 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 18:47:56 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:56 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:56.368543 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:56.363147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:33467)
0619 18:47:56.363437 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:56.363452 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:56.363585 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:56.363861 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:56.363863 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:56.364088 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:56.364233 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:56.364239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.364967 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.364971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:56.365536 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:56.365544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.365964 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.365967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:56.366049 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:56.366566 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:56.366578 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:56.368359 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
W20260619 18:47:56.368610 22412 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:56.363297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:33467 (local address 127.12.234.254:40337)
0619 18:47:56.363439 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:56.363443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:56.363529 (+ 86us) server_negotiation.cc:407] Connection header received
0619 18:47:56.363691 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:56.363694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:56.363740 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:56.363822 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:56.364350 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.364868 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.365650 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.365866 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.365950 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:56.368505 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:33467: BlockingRecv error: recv got EOF from 127.12.234.195:33467 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 18:47:56 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:56 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:56.989012 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:56.983368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:60909)
0619 18:47:56.983712 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:56.983730 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:56.983866 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:56.984283 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:56.984286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:56.984459 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:56.984559 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:56.984564 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.985330 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.985333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:56.985909 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:56.985917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.986460 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.986464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:56.986543 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:56.987106 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:56.987118 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:56.988893 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":99,"threads_started":1}
W20260619 18:47:56.989187 22416 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:56.983462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:60909 (local address 127.12.234.254:40337)
0619 18:47:56.983740 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:56.983744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:56.983814 (+ 70us) server_negotiation.cc:407] Connection header received
0619 18:47:56.983998 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:56.984001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:56.984069 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:56.984155 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:56.984673 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.985212 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.986034 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:56.986306 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:56.986394 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:56.989034 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:60909: BlockingRecv error: recv got EOF from 127.12.234.193:60909 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
Jun 19 18:47:57 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:57 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:57.182090 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:57.176730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:37627)
0619 18:47:57.176974 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:57.176987 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:57.177094 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:57.177322 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:57.177324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:57.177492 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:57.177589 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:57.177593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.178413 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.178416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:57.178995 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:57.179024 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.179500 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.179503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:57.179588 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:57.180085 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:57.180098 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:57.181956 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
W20260619 18:47:57.182219 22416 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:57.176823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:37627 (local address 127.12.234.254:40337)
0619 18:47:57.176988 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:57.176992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:57.177044 (+ 52us) server_negotiation.cc:407] Connection header received
0619 18:47:57.177179 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:57.177186 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:57.177231 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:57.177298 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:57.177677 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.178309 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.179132 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.179392 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.179483 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:57.182077 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:37627: BlockingRecv error: recv got EOF from 127.12.234.194:37627 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 19 18:47:57 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:57 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:57.374318 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:57.369283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:44681)
0619 18:47:57.369528 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:57.369541 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:57.369637 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:57.369841 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:57.369843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:57.370011 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:57.370105 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:57.370110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.370880 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.370884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:57.371472 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:57.371480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.371876 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.371879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:57.371949 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:57.372441 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:57.372453 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:57.374159 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":81,"threads_started":1}
W20260619 18:47:57.374461 22416 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:57.369363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:44681 (local address 127.12.234.254:40337)
0619 18:47:57.369512 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:57.369516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:57.369600 (+ 84us) server_negotiation.cc:407] Connection header received
0619 18:47:57.369705 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:57.369707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:57.369751 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:57.369818 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:57.370208 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.370751 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.371565 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.371769 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.371838 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:57.374336 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:44681: BlockingRecv error: recv got EOF from 127.12.234.195:44681 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 18:47:57 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:57 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:57.999013 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:57.989877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:40603)
0619 18:47:57.990252 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:57.990269 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:57.990389 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:57.990685 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:57.990688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:57.990936 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:57.991079 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:57.991089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.991892 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.991895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:57.992459 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:57.992467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.992879 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.992882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:57.992945 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:57.997061 (+ 4116us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:57.997075 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:57.998890 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":79,"threads_started":1}
W20260619 18:47:57.999198 22420 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:57.989975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:40603 (local address 127.12.234.254:40337)
0619 18:47:57.990236 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:57.990241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:57.990354 (+ 113us) server_negotiation.cc:407] Connection header received
0619 18:47:57.990485 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:57.990488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:57.990544 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:57.990636 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:57.991202 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.991784 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.992553 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:57.992793 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:57.992864 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:57.999062 (+ 6198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:40603: BlockingRecv error: recv got EOF from 127.12.234.193:40603 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":74,"threads_started":1}
Jun 19 18:47:58 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:58 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:58.188524 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:58.182863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:59365)
0619 18:47:58.183145 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:58.183161 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:58.183300 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:58.183529 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:58.183532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:58.183725 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:58.183817 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:58.183822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:58.184571 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:58.184575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:58.185145 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:58.185153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:58.185587 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:58.185591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:58.185672 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:58.186366 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:58.186379 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:58.188392 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":95,"threads_started":1}
W20260619 18:47:58.188632 22420 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:58.182894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:59365 (local address 127.12.234.254:40337)
0619 18:47:58.183011 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:58.183015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:58.183267 (+ 252us) server_negotiation.cc:407] Connection header received
0619 18:47:58.183389 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:58.183392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:58.183440 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:58.183498 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:58.183942 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:58.184470 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:58.185262 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:58.185489 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:58.185573 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:58.188534 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:59365: BlockingRecv error: recv got EOF from 127.12.234.194:59365 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 19 18:47:58 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:58 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:58.380685 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:58.375055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:45771)
0619 18:47:58.375333 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:58.375351 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:58.375462 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:58.375742 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:58.375745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:58.375947 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:58.376050 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:58.376059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:58.376826 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:58.376830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:58.377395 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:58.377403 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:58.377849 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:58.377853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:58.377935 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:58.378534 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:58.378568 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:58.380506 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":74,"threads_started":1}
W20260619 18:47:58.380776 22420 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:58.375280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:45771 (local address 127.12.234.254:40337)
0619 18:47:58.375401 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:58.375406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:58.375421 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:58.375579 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:58.375582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:58.375632 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:58.375706 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:58.376174 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:58.376712 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:58.377512 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:58.377741 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:58.377836 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:58.380661 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:45771: BlockingRecv error: recv got EOF from 127.12.234.195:45771 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 19 18:47:59 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:59 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:59.005311 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:58.999861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:46735)
0619 18:47:59.000129 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:59.000142 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:59.000283 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:59.000539 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:59.000541 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:59.000755 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:59.000878 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:59.000887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.001870 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.001874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:59.002455 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:59.002464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.002858 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.002860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:59.002915 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:59.003411 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:59.003423 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:59.005151 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":85,"threads_started":1}
W20260619 18:47:59.005403 22424 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:58.999945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:46735 (local address 127.12.234.254:40337)
0619 18:47:59.000178 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:59.000181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:59.000196 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:47:59.000370 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:59.000372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:59.000416 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:59.000494 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:59.001003 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.001731 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.002552 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.002767 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.002850 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:59.005291 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:46735: BlockingRecv error: recv got EOF from 127.12.234.193:46735 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":61,"threads_started":1}
Jun 19 18:47:59 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:59 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:59.194388 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:59.189391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:38959)
0619 18:47:59.189710 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:59.189724 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:59.189831 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:59.190083 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:59.190085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:59.190337 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:59.190461 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:59.190469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.191153 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.191155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:59.191696 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:59.191703 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.192076 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.192079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:59.192146 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:59.192632 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:59.192643 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:59.194270 (+ 1627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":76,"threads_started":1}
W20260619 18:47:59.194516 22424 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:59.189435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:38959 (local address 127.12.234.254:40337)
0619 18:47:59.189553 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:59.189557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:59.189798 (+ 241us) server_negotiation.cc:407] Connection header received
0619 18:47:59.189903 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:59.189906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:59.189960 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:59.190034 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:59.190560 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.191070 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.191777 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.191986 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.192054 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:59.194402 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:38959: BlockingRecv error: recv got EOF from 127.12.234.194:38959 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 19 18:47:59 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:47:59 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:47:59.387012 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:47:59.381573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:52591)
0619 18:47:59.381885 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 18:47:59.381902 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:47:59.382025 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:47:59.382318 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:47:59.382321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:47:59.382540 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:47:59.382634 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:59.382639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.383428 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.383432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:59.383983 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:47:59.383992 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.384414 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.384417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:47:59.384496 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:59.384984 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:47:59.384995 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:47:59.386848 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":81,"threads_started":1}
W20260619 18:47:59.387107 22424 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:47:59.381658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:52591 (local address 127.12.234.254:40337)
0619 18:47:59.381834 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 18:47:59.381838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:47:59.381991 (+ 153us) server_negotiation.cc:407] Connection header received
0619 18:47:59.382114 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:47:59.382118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:47:59.382172 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:47:59.382279 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:47:59.382743 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.383326 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.384088 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:47:59.384312 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:47:59.384403 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:47:59.387006 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:52591: BlockingRecv error: recv got EOF from 127.12.234.195:52591 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 19 18:48:00 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:00 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:00.011642 22427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:00.006135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:50369)
0619 18:48:00.006437 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:00.006450 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:00.006545 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:00.006782 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:00.006785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:00.006993 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:00.007093 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:00.007098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.007866 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.007870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:00.008526 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:00.008535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.008994 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.008997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:00.009091 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:00.009662 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:00.009676 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:00.011507 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":105,"threads_started":1}
W20260619 18:48:00.011781 22428 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:00.006281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:50369 (local address 127.12.234.254:40337)
0619 18:48:00.006509 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:00.006513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:00.006528 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:48:00.006624 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:00.006626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:00.006672 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:00.006761 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:00.007195 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.007758 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.008644 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.008896 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.008974 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:00.011642 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:50369: BlockingRecv error: recv got EOF from 127.12.234.193:50369 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":66,"threads_started":1}
Jun 19 18:48:00 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:00 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:00.200654 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:00.195170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:38211)
0619 18:48:00.195497 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:00.195513 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:00.195635 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:00.195925 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:00.195928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:00.196103 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:00.196205 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:00.196210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.196913 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.196915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:00.197480 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:00.197487 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.198007 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.198012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:00.198093 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:00.198656 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:00.198671 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:00.200507 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":112,"threads_started":1}
W20260619 18:48:00.200744 22428 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:00.195288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:38211 (local address 127.12.234.254:40337)
0619 18:48:00.195437 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:00.195442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:00.195586 (+ 144us) server_negotiation.cc:407] Connection header received
0619 18:48:00.195723 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:00.195726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:00.195773 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:00.195862 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:00.196297 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.196833 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.197587 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.197896 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.197977 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:00.200645 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:38211: BlockingRecv error: recv got EOF from 127.12.234.194:38211 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 18:48:00 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:00 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:00.393254 22430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:00.387949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:39049)
0619 18:48:00.388224 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:00.388237 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:00.388356 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:00.388612 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:00.388615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:00.388840 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:00.388948 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:00.388954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.389762 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.389764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:00.390354 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:00.390363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.390828 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.390831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:00.390903 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:00.391390 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:00.391401 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:00.393124 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":84,"threads_started":1}
W20260619 18:48:00.393441 22428 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:00.388003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:39049 (local address 127.12.234.254:40337)
0619 18:48:00.388126 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:00.388130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:00.388313 (+ 183us) server_negotiation.cc:407] Connection header received
0619 18:48:00.388443 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:00.388446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:00.388496 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:00.388567 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:00.389088 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.389632 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.390467 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:00.390694 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:00.390790 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:00.393320 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:39049: BlockingRecv error: recv got EOF from 127.12.234.195:39049 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 18:48:01 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:01 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:01.017900 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:01.012542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:35609)
0619 18:48:01.012855 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:01.012869 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:01.012970 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:01.013291 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:01.013293 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:01.013471 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:01.013567 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:01.013572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.014392 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.014395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:01.014960 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:01.014969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.015422 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.015425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:01.015485 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:01.015966 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:01.015978 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:01.017733 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":101,"threads_started":1}
W20260619 18:48:01.018033 22432 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:01.012685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:35609 (local address 127.12.234.254:40337)
0619 18:48:01.012907 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:01.012910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:01.012926 (+ 16us) server_negotiation.cc:407] Connection header received
0619 18:48:01.013096 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:01.013099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:01.013146 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:01.013228 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:01.013693 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.014289 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.015097 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.015326 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.015409 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:01.017900 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:35609: BlockingRecv error: recv got EOF from 127.12.234.193:35609 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":60,"threads_started":1}
Jun 19 18:48:01 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:01 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:01.207122 22433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:01.201380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:47905)
0619 18:48:01.201636 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:01.201653 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:01.201762 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:01.202017 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:01.202020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:01.202261 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:01.202380 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:01.202386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.203147 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.203150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:01.203723 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:01.203732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.204170 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.204174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:01.204253 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:01.204819 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:01.204839 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:01.206954 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":78,"threads_started":1}
W20260619 18:48:01.207254 22432 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:01.201547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:47905 (local address 127.12.234.254:40337)
0619 18:48:01.201692 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:01.201696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:01.201711 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:48:01.201863 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:01.201867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:01.201913 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:01.201977 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:01.202504 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.203037 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.203850 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.204066 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.204152 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:01.207125 (+ 2973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:47905: BlockingRecv error: recv got EOF from 127.12.234.194:47905 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 18:48:01 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:01 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:01.399791 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:01.394113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:44233)
0619 18:48:01.394425 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:01.394440 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:01.394569 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:01.394830 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:01.394832 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:01.395033 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:01.395138 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:01.395145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.395907 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.395910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:01.396492 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:01.396501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.396960 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.396963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:01.397047 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:01.397553 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:01.397565 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:01.399635 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":104,"threads_started":1}
W20260619 18:48:01.399856 22432 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:01.394254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:44233 (local address 127.12.234.254:40337)
0619 18:48:01.394425 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:01.394430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:01.394515 (+ 85us) server_negotiation.cc:407] Connection header received
0619 18:48:01.394673 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:01.394676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:01.394722 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:01.394789 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:01.395265 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.395792 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.396619 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:01.396846 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:01.396934 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:01.399757 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:44233: BlockingRecv error: recv got EOF from 127.12.234.195:44233 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 19 18:48:02 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:02 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:02.024410 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:02.018752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:45703)
0619 18:48:02.019017 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:02.019031 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:02.019128 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:02.019378 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:02.019380 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:02.019557 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:02.019654 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:02.019659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.020416 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.020419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:02.020981 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:02.020989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.021515 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.021517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:02.021578 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:02.022076 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:02.022088 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:02.024258 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
W20260619 18:48:02.024672 22436 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:02.018859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:45703 (local address 127.12.234.254:40337)
0619 18:48:02.019147 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:02.019151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:02.019182 (+ 31us) server_negotiation.cc:407] Connection header received
0619 18:48:02.019231 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:02.019233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:02.019277 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:02.019351 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:02.019749 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.020315 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.021171 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.021407 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.021502 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:02.024441 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:45703: BlockingRecv error: recv got EOF from 127.12.234.193:45703 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":63,"threads_started":1}
Jun 19 18:48:02 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:02 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:02.214538 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:02.207891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:41885)
0619 18:48:02.208238 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:02.208250 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:02.208370 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:02.209252 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:02.209255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:02.209476 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:02.209597 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:02.209606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.210602 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.210606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:02.211435 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:02.211445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.211897 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.211900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:02.211960 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:02.212423 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:02.212435 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:02.214375 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":96,"threads_started":1}
W20260619 18:48:02.214632 22436 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:02.207984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:41885 (local address 127.12.234.254:40337)
0619 18:48:02.208117 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:02.208122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:02.208328 (+ 206us) server_negotiation.cc:407] Connection header received
0619 18:48:02.208489 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:02.208492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:02.208548 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:02.208639 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:02.209783 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.210487 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.211566 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.211792 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.211888 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:02.214526 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:41885: BlockingRecv error: recv got EOF from 127.12.234.194:41885 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 18:48:02 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:02 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.195@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:02.406567 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:02.400659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.195:52819)
0619 18:48:02.400928 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:02.400941 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:02.401051 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:02.401432 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:02.401435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:02.401623 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:02.401727 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:02.401734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.402621 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.402624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:02.403213 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:02.403221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.403688 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.403691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:02.403751 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:02.404302 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:02.404315 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:02.406424 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":82,"threads_started":1}
W20260619 18:48:02.406720 22436 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:02.400809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:52819 (local address 127.12.234.254:40337)
0619 18:48:02.400943 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:02.400947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:02.401014 (+ 67us) server_negotiation.cc:407] Connection header received
0619 18:48:02.401215 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:02.401218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:02.401271 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:02.401354 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:02.401877 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.402507 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.403360 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:02.403590 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:02.403677 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:02.406580 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.195:52819: BlockingRecv error: recv got EOF from 127.12.234.195:52819 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 18:48:03 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:03 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.193@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:03.036175 22439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:03.025502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.193:34479)
0619 18:48:03.025772 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:03.025794 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:03.025906 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:03.026387 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:03.026390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:03.026573 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:03.026674 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:03.026680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.027597 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.027601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:03.028218 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:03.028229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.028785 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.028788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:03.028863 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:03.034016 (+ 5153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:03.034031 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:03.036049 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
W20260619 18:48:03.036391 22440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:03.025658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:34479 (local address 127.12.234.254:40337)
0619 18:48:03.025998 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:03.026001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:03.026017 (+ 16us) server_negotiation.cc:407] Connection header received
0619 18:48:03.026067 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:03.026075 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:03.026116 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:03.026303 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:03.026815 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.027448 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.028405 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.028639 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.028737 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:03.036258 (+ 7521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.193:34479: BlockingRecv error: recv got EOF from 127.12.234.193:34479 (error 108)
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":91,"threads_started":1}
Jun 19 18:48:03 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 18:48:03 dist-test-slave-fljv krb5kdc[21381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.194@KRBTEST.COM for oryx/127.12.234.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 18:48:03.220933 22441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:03.215580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40337 (local address 127.12.234.194:33983)
0619 18:48:03.215836 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:03.215851 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:03.215946 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:03.216180 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:03.216182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:03.216354 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:03.216446 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:03.216451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.217295 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.217306 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:03.217877 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:03.217887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.218380 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.218384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:03.218473 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:03.218959 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:03.218970 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:03.220731 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.254:40337: Server oryx/127.12.234.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":69,"threads_started":1}
W20260619 18:48:03.220991 22440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:03.215656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:33983 (local address 127.12.234.254:40337)
0619 18:48:03.215826 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:03.215830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:03.215913 (+ 83us) server_negotiation.cc:407] Connection header received
0619 18:48:03.216012 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:03.216014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:03.216056 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:03.216127 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:03.216578 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.217159 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.217992 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.218266 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.218365 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:03.220885 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.194:33983: BlockingRecv error: recv got EOF from 127.12.234.194:33983 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 18:48:03.314890 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 21964
I20260619 18:48:03.319741 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 22116
I20260619 18:48:03.324293 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 22249
I20260619 18:48:03.328722 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 21894
2026-06-19T18:48:03Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17862 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T18:48:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:48:03Z Disabled control of system clock
I20260619 18:48:03.357501 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:37181
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:41479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:37181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 18:48:03.468201 22445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:03.468479 22445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:03.468513 22445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:03.472565 22445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:48:03.472647 22445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:03.472669 22445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:03.472687 22445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:48:03.472700 22445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:48:03.477568 22445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:37181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:37181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:03.478631 22445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:03.479980 22445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:03.489138 22450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:03.489195 22453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:03.489195 22451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:03.489300 22445 server_base.cc:1034] running on GCE node
I20260619 18:48:03.490123 22445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:03.491389 22445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:03.492573 22445 hybrid_clock.cc:648] HybridClock initialized: now 1781894883492544 us; error 48 us; skew 500 ppm
I20260619 18:48:03.494575 22445 webserver.cc:460] Webserver started at http://127.12.234.254:37379/ using document root <none> and password file <none>
I20260619 18:48:03.495180 22445 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:03.495250 22445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:03.495440 22445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:03.497539 22445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "a666d6e2789649a0ab72e388f77dcb50"
format_stamp: "Formatted at 2026-06-19 18:48:03 on dist-test-slave-fljv"
server_key: "2383c6760a2fdc564460eb7d7ef4784a"
server_key_iv: "b9e06e7ce15472b48c47c105b53f17ee"
server_key_version: "encryptionkey@0"
I20260619 18:48:03.498106 22445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "a666d6e2789649a0ab72e388f77dcb50"
format_stamp: "Formatted at 2026-06-19 18:48:03 on dist-test-slave-fljv"
server_key: "2383c6760a2fdc564460eb7d7ef4784a"
server_key_iv: "b9e06e7ce15472b48c47c105b53f17ee"
server_key_version: "encryptionkey@0"
I20260619 18:48:03.501816 22445 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:48:03.504429 22459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:03.505795 22445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:48:03.505930 22445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "a666d6e2789649a0ab72e388f77dcb50"
format_stamp: "Formatted at 2026-06-19 18:48:03 on dist-test-slave-fljv"
server_key: "2383c6760a2fdc564460eb7d7ef4784a"
server_key_iv: "b9e06e7ce15472b48c47c105b53f17ee"
server_key_version: "encryptionkey@0"
I20260619 18:48:03.506037 22445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:03.522480 22445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:03.523260 22445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:03.523460 22445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:03.530601 22445 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:37181
I20260619 18:48:03.530612 22511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:37181 every 8 connection(s)
I20260619 18:48:03.531649 22445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:48:03.533842 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 22445
I20260619 18:48:03.533969 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:48:03.534328 13227 external_mini_cluster.cc:1428] Setting key 09a9ec5c2005f67c6e4ac15754de5260
I20260619 18:48:03.535461 22512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:03.541939 22512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50: Bootstrap starting.
I20260619 18:48:03.544323 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:03.535877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50888 (local address 127.12.234.254:37181)
0619 18:48:03.536507 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:03.536525 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:03.536563 (+ 38us) server_negotiation.cc:407] Connection header received
0619 18:48:03.537269 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:03.537289 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:03.537720 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:03.538027 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:03.538461 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.539477 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.540711 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.541240 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.541375 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:03.542379 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:03.542385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:03.542398 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:48:03.542816 (+ 418us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:03.542873 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:48:03.542894 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:03.542973 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:03.543270 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 18:48:03.543559 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":123,"threads_started":1}
I20260619 18:48:03.544377 22512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:03.545130 22512 log.cc:824] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:03.547173 22512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50: No bootstrap required, opened a new log
I20260619 18:48:03.550812 22512 raft_consensus.cc:348] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a666d6e2789649a0ab72e388f77dcb50" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37181 } }
I20260619 18:48:03.551074 22512 raft_consensus.cc:374] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:03.551136 22512 raft_consensus.cc:729] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a666d6e2789649a0ab72e388f77dcb50, State: Initialized, Role: FOLLOWER
I20260619 18:48:03.551600 22512 consensus_queue.cc:260] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a666d6e2789649a0ab72e388f77dcb50" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37181 } }
I20260619 18:48:03.551750 22512 raft_consensus.cc:388] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:48:03.551820 22512 raft_consensus.cc:482] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:48:03.551934 22512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:03.552901 22512 raft_consensus.cc:504] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a666d6e2789649a0ab72e388f77dcb50" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37181 } }
I20260619 18:48:03.553247 22512 leader_election.cc:302] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a666d6e2789649a0ab72e388f77dcb50; no voters:
I20260619 18:48:03.553555 22512 leader_election.cc:288] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:48:03.553682 22517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:48:03.553915 22517 raft_consensus.cc:686] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [term 1 LEADER]: Becoming Leader. State: Replica: a666d6e2789649a0ab72e388f77dcb50, State: Running, Role: LEADER
I20260619 18:48:03.554366 22517 consensus_queue.cc:237] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a666d6e2789649a0ab72e388f77dcb50" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37181 } }
I20260619 18:48:03.554833 22512 sys_catalog.cc:564] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:48:03.556044 22518 sys_catalog.cc:455] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a666d6e2789649a0ab72e388f77dcb50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a666d6e2789649a0ab72e388f77dcb50" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37181 } } }
I20260619 18:48:03.556072 22519 sys_catalog.cc:455] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a666d6e2789649a0ab72e388f77dcb50. Latest consensus state: current_term: 1 leader_uuid: "a666d6e2789649a0ab72e388f77dcb50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a666d6e2789649a0ab72e388f77dcb50" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37181 } } }
I20260619 18:48:03.556165 22518 sys_catalog.cc:458] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [sys.catalog]: This master's current role is: LEADER
I20260619 18:48:03.556172 22519 sys_catalog.cc:458] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50 [sys.catalog]: This master's current role is: LEADER
W20260619 18:48:03.559271 22532 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:48:03.559345 22532 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:48:03.559417 22526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:48:03.560168 22526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:48:03.565737 22526 catalog_manager.cc:1347] Generated new cluster ID: 1287d90a9b9647808d615244781168b7
I20260619 18:48:03.565809 22526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:48:03.572504 22526 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:48:03.573396 22526 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:48:03.584123 22526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a666d6e2789649a0ab72e388f77dcb50: Generated new TSK 0
I20260619 18:48:03.584862 22526 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:48:03.591501 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37181
--builtin_ntp_servers=127.12.234.212:41479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 18:48:03.705363 22536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:03.705672 22536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:03.705711 22536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:03.709944 22536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:03.710019 22536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:03.710106 22536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:48:03.715108 22536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:03.716411 22536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:03.717954 22536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:03.728030 22542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:03.728060 22544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:03.728024 22541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:03.728407 22536 server_base.cc:1034] running on GCE node
I20260619 18:48:03.728844 22536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:03.729506 22536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:03.730666 22536 hybrid_clock.cc:648] HybridClock initialized: now 1781894883730639 us; error 46 us; skew 500 ppm
I20260619 18:48:03.732493 22536 webserver.cc:460] Webserver started at http://127.12.234.193:45061/ using document root <none> and password file <none>
I20260619 18:48:03.733063 22536 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:03.733127 22536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:03.733318 22536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:03.735417 22536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "9c974ac81041415589da690162b550b9"
format_stamp: "Formatted at 2026-06-19 18:48:03 on dist-test-slave-fljv"
server_key: "e29de27dbd0b95339bd53c1978d4d7eb"
server_key_iv: "d105e479a05b23e74d3a4887003762c8"
server_key_version: "encryptionkey@0"
I20260619 18:48:03.735966 22536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "9c974ac81041415589da690162b550b9"
format_stamp: "Formatted at 2026-06-19 18:48:03 on dist-test-slave-fljv"
server_key: "e29de27dbd0b95339bd53c1978d4d7eb"
server_key_iv: "d105e479a05b23e74d3a4887003762c8"
server_key_version: "encryptionkey@0"
I20260619 18:48:03.739820 22536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:48:03.742574 22550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:03.743876 22536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:48:03.743994 22536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "9c974ac81041415589da690162b550b9"
format_stamp: "Formatted at 2026-06-19 18:48:03 on dist-test-slave-fljv"
server_key: "e29de27dbd0b95339bd53c1978d4d7eb"
server_key_iv: "d105e479a05b23e74d3a4887003762c8"
server_key_version: "encryptionkey@0"
I20260619 18:48:03.744108 22536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:03.754828 22536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:03.755570 22536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:03.755744 22536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:03.756331 22536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:03.757339 22536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:03.757386 22536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:03.757440 22536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:03.757470 22536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:03.766075 22536 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:34907
I20260619 18:48:03.766111 22663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:34907 every 8 connection(s)
I20260619 18:48:03.767076 22536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:48:03.767874 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 22536
I20260619 18:48:03.767987 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:48:03.768311 13227 external_mini_cluster.cc:1428] Setting key c8b7c8579721bf19b1ff163352fefdc1
I20260619 18:48:03.770292 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37181
--builtin_ntp_servers=127.12.234.212:41479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 18:48:03.777463 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:03.769111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:40597 (local address 127.12.234.254:37181)
0619 18:48:03.769236 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:03.769240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:03.770473 (+ 1233us) server_negotiation.cc:407] Connection header received
0619 18:48:03.771888 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:03.771891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:03.771952 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:03.772028 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:03.773185 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.773760 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.775493 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.775722 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.775801 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:03.777024 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:03.777027 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:03.777029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:48:03.777042 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:03.777060 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:48:03.777067 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:03.777120 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:03.777290 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 18:48:03.777351 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 18:48:03.778728 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:03.769562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:37181 (local address 127.12.234.193:40597)
0619 18:48:03.770332 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:03.770385 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:03.771668 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:03.772232 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:03.772239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:03.772364 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:03.773029 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:03.773048 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.773905 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.773909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:03.775376 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:03.775388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.775884 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.775888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:03.776100 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:03.776721 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:48:03.776723 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:03.776783 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:48:03.776871 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:03.777146 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:03.777154 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:03.777160 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:48:03.777294 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 18:48:03.777621 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":119,"threads_started":1}
I20260619 18:48:03.779964 22664 heartbeater.cc:347] Connected to a master server at 127.12.234.254:37181
I20260619 18:48:03.780237 22664 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:03.780877 22664 heartbeater.cc:511] Master 127.12.234.254:37181 requested a full tablet report, sending...
I20260619 18:48:03.782501 22475 ts_manager.cc:194] Registered new tserver with Master: 9c974ac81041415589da690162b550b9 (127.12.234.193:34907)
I20260619 18:48:03.783612 22475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.193:40597
W20260619 18:48:03.883906 22667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:03.884212 22667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:03.884265 22667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:03.888684 22667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:03.888751 22667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:03.888828 22667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:48:03.893749 22667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:03.894917 22667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:03.896359 22667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:03.906261 22673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:03.906231 22672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:03.906360 22675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:03.906468 22667 server_base.cc:1034] running on GCE node
I20260619 18:48:03.906986 22667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:03.907625 22667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:03.908792 22667 hybrid_clock.cc:648] HybridClock initialized: now 1781894883908768 us; error 34 us; skew 500 ppm
I20260619 18:48:03.910709 22667 webserver.cc:460] Webserver started at http://127.12.234.194:37009/ using document root <none> and password file <none>
I20260619 18:48:03.911273 22667 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:03.911350 22667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:03.911551 22667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:03.913595 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "92c735d2862346a38b70f0c7f3f4c964"
format_stamp: "Formatted at 2026-06-19 18:48:03 on dist-test-slave-fljv"
server_key: "c442e332c0d7c4d5938e1cf64e9d78c0"
server_key_iv: "be61de05baf7e603cb035f88d9937ce5"
server_key_version: "encryptionkey@0"
I20260619 18:48:03.914126 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "92c735d2862346a38b70f0c7f3f4c964"
format_stamp: "Formatted at 2026-06-19 18:48:03 on dist-test-slave-fljv"
server_key: "c442e332c0d7c4d5938e1cf64e9d78c0"
server_key_iv: "be61de05baf7e603cb035f88d9937ce5"
server_key_version: "encryptionkey@0"
I20260619 18:48:03.917995 22667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:48:03.920735 22681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:03.922067 22667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:48:03.922214 22667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "92c735d2862346a38b70f0c7f3f4c964"
format_stamp: "Formatted at 2026-06-19 18:48:03 on dist-test-slave-fljv"
server_key: "c442e332c0d7c4d5938e1cf64e9d78c0"
server_key_iv: "be61de05baf7e603cb035f88d9937ce5"
server_key_version: "encryptionkey@0"
I20260619 18:48:03.922336 22667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:03.955574 22667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:03.956368 22667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:03.956552 22667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:03.957158 22667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:03.958113 22667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:03.958164 22667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:03.958240 22667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:03.958267 22667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:03.966607 22667 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:39793
I20260619 18:48:03.966648 22794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:39793 every 8 connection(s)
I20260619 18:48:03.967564 22667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:48:03.976749 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:03.969337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:45393 (local address 127.12.234.254:37181)
0619 18:48:03.969485 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:03.969489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:03.970740 (+ 1251us) server_negotiation.cc:407] Connection header received
0619 18:48:03.972119 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:03.972121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:03.972185 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:03.972262 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:03.973267 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.973797 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.975007 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.975256 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.975340 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:03.976359 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:03.976363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:03.976365 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:48:03.976377 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:03.976397 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:48:03.976404 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:03.976458 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:03.976569 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 18:48:03.976626 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 18:48:03.977181 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 22667
I20260619 18:48:03.977278 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:48:03.977551 13227 external_mini_cluster.cc:1428] Setting key ee68c918eafdeeffb9a436dc64b752ea
I20260619 18:48:03.977660 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:03.969782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:37181 (local address 127.12.234.194:45393)
0619 18:48:03.970565 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:03.970609 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:03.971910 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:03.972441 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:03.972449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:03.972539 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:03.973079 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:03.973092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.973943 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.973946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:03.974888 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:03.974898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:03.975384 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:03.975387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:03.975529 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:03.976102 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:48:03.976104 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:03.976146 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:48:03.976213 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:03.976466 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:03.976471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:03.976475 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 18:48:03.976563 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 18:48:03.976793 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":153,"threads_started":1}
I20260619 18:48:03.978914 22795 heartbeater.cc:347] Connected to a master server at 127.12.234.254:37181
I20260619 18:48:03.979168 22795 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:03.979801 22795 heartbeater.cc:511] Master 127.12.234.254:37181 requested a full tablet report, sending...
I20260619 18:48:03.979897 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37181
--builtin_ntp_servers=127.12.234.212:41479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 18:48:03.980937 22476 ts_manager.cc:194] Registered new tserver with Master: 92c735d2862346a38b70f0c7f3f4c964 (127.12.234.194:39793)
I20260619 18:48:03.981554 22476 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.194:45393
W20260619 18:48:04.092010 22798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:04.092296 22798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:04.092332 22798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:04.096479 22798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:04.096539 22798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:04.096597 22798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:48:04.101598 22798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:04.102888 22798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:04.104256 22798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:48:04.113960 22798 server_base.cc:1034] running on GCE node
W20260619 18:48:04.113906 22806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:04.113912 22803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:04.113917 22804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:04.114638 22798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:04.115273 22798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:04.116438 22798 hybrid_clock.cc:648] HybridClock initialized: now 1781894884116396 us; error 60 us; skew 500 ppm
I20260619 18:48:04.118367 22798 webserver.cc:460] Webserver started at http://127.12.234.195:46295/ using document root <none> and password file <none>
I20260619 18:48:04.118952 22798 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:04.119021 22798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:04.119212 22798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:04.121328 22798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "15a2d15803ed40d2ac27319ae37771ec"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "a7569837f2acf4d61164fcb0f304242e"
server_key_iv: "f5adce8a36f63259d577570560c02301"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.121867 22798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "15a2d15803ed40d2ac27319ae37771ec"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "a7569837f2acf4d61164fcb0f304242e"
server_key_iv: "f5adce8a36f63259d577570560c02301"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.125742 22798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:48:04.128572 22812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.129885 22798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 18:48:04.130005 22798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "15a2d15803ed40d2ac27319ae37771ec"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "a7569837f2acf4d61164fcb0f304242e"
server_key_iv: "f5adce8a36f63259d577570560c02301"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.130117 22798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:04.148710 22798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:04.149448 22798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:04.149624 22798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:04.150257 22798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:04.151202 22798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:04.151258 22798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.151310 22798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:04.151340 22798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.160084 22798 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:45603
I20260619 18:48:04.160108 22925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:45603 every 8 connection(s)
I20260619 18:48:04.161015 22798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:48:04.167548 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 22798
I20260619 18:48:04.167665 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:48:04.167982 13227 external_mini_cluster.cc:1428] Setting key 8d7cb21dd886defc3b4ed69ad92e0e04
I20260619 18:48:04.170524 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:04.162833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:42495 (local address 127.12.234.254:37181)
0619 18:48:04.162947 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:04.162951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:04.164030 (+ 1079us) server_negotiation.cc:407] Connection header received
0619 18:48:04.165460 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:04.165463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:04.165515 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:04.165595 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:04.166600 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:04.167137 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:04.168624 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:04.168905 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:04.168994 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:04.170073 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:04.170077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:04.170080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:48:04.170094 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:04.170115 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:48:04.170123 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:04.170212 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:04.170361 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 18:48:04.170414 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 18:48:04.171380 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:04.163228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:37181 (local address 127.12.234.195:42495)
0619 18:48:04.163878 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:04.163919 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:04.165189 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:04.165765 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:04.165772 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:04.165862 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:04.166453 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:04.166469 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:04.167261 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:04.167264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:04.168505 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:04.168515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:04.169051 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:04.169055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:04.169211 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:04.169776 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:48:04.169778 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:04.169835 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:48:04.169919 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:04.170233 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:04.170241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:04.170246 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 18:48:04.170351 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 18:48:04.170592 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
I20260619 18:48:04.172385 22926 heartbeater.cc:347] Connected to a master server at 127.12.234.254:37181
I20260619 18:48:04.172603 22926 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:04.173105 22926 heartbeater.cc:511] Master 127.12.234.254:37181 requested a full tablet report, sending...
I20260619 18:48:04.174028 22476 ts_manager.cc:194] Registered new tserver with Master: 15a2d15803ed40d2ac27319ae37771ec (127.12.234.195:45603)
I20260619 18:48:04.174588 22476 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.195:42495
I20260619 18:48:04.181854 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 18:48:04.184592 22515 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:04.183685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50898 (local address 127.12.234.254:37181)
0619 18:48:04.183814 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:04.183819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:04.183977 (+ 158us) server_negotiation.cc:407] Connection header received
0619 18:48:04.184090 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:04.184093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:04.184137 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:04.184202 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:04.184455 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50898: BlockingRecv error: recv got EOF from 127.0.0.1:50898 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:48:04.184715 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 22536
I20260619 18:48:04.189301 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 22667
I20260619 18:48:04.193677 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 22798
I20260619 18:48:04.198274 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 22445
2026-06-19T18:48:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (869 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T18:48:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:48:04Z Disabled control of system clock
I20260619 18:48:04.226964 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41455
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:41455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 18:48:04.336323 22937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:04.336632 22937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:04.336668 22937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:04.340823 22937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:48:04.340905 22937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:04.340919 22937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:04.340935 22937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:48:04.340950 22937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:48:04.346014 22937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:41455
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:04.347271 22937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:04.348703 22937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:04.357486 22942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:04.357498 22937 server_base.cc:1034] running on GCE node
W20260619 18:48:04.357506 22945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:04.357563 22943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:04.358376 22937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:04.359529 22937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:04.360692 22937 hybrid_clock.cc:648] HybridClock initialized: now 1781894884360672 us; error 35 us; skew 500 ppm
I20260619 18:48:04.362593 22937 webserver.cc:460] Webserver started at http://127.12.234.254:34637/ using document root <none> and password file <none>
I20260619 18:48:04.363152 22937 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:04.363219 22937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:04.363415 22937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:04.365453 22937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "f24b15e355d64db8b036a77d11ef213c"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "1cef22a76ed673788a98681a7463645d"
server_key_iv: "e599e38f4f2ca517c7ba76a59f18d8d1"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.365993 22937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "f24b15e355d64db8b036a77d11ef213c"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "1cef22a76ed673788a98681a7463645d"
server_key_iv: "e599e38f4f2ca517c7ba76a59f18d8d1"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.369791 22937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:48:04.372344 22951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.373611 22937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 18:48:04.373746 22937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "f24b15e355d64db8b036a77d11ef213c"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "1cef22a76ed673788a98681a7463645d"
server_key_iv: "e599e38f4f2ca517c7ba76a59f18d8d1"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.373858 22937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:04.378507 22937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:04.379127 22937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:04.379288 22937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:04.385622 22937 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:41455
I20260619 18:48:04.385654 23003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:41455 every 8 connection(s)
I20260619 18:48:04.386638 22937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:48:04.389904 23004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:04.392302 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 22937
I20260619 18:48:04.392405 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:48:04.392663 13227 external_mini_cluster.cc:1428] Setting key 36c5088d44fc5952a0b242305e494e77
I20260619 18:48:04.395934 23004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c: Bootstrap starting.
I20260619 18:48:04.398402 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:04.394152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.12.234.254:41455)
0619 18:48:04.394867 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:04.394884 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:04.394920 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:48:04.395560 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:04.395578 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:04.395959 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:04.396299 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:04.397070 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:04.397075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:04.397087 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:48:04.397097 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:04.397152 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:48:04.397169 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:04.397293 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:04.397389 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 18:48:04.397630 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":125,"threads_started":1}
I20260619 18:48:04.398411 23004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:04.399323 23004 log.cc:824] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:04.401401 23004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c: No bootstrap required, opened a new log
I20260619 18:48:04.404852 23004 raft_consensus.cc:348] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f24b15e355d64db8b036a77d11ef213c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41455 } }
I20260619 18:48:04.405107 23004 raft_consensus.cc:374] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:04.405176 23004 raft_consensus.cc:729] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24b15e355d64db8b036a77d11ef213c, State: Initialized, Role: FOLLOWER
I20260619 18:48:04.405652 23004 consensus_queue.cc:260] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f24b15e355d64db8b036a77d11ef213c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41455 } }
I20260619 18:48:04.405840 23004 raft_consensus.cc:388] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:48:04.405915 23004 raft_consensus.cc:482] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:48:04.406014 23004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:04.407058 23004 raft_consensus.cc:504] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f24b15e355d64db8b036a77d11ef213c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41455 } }
I20260619 18:48:04.407397 23004 leader_election.cc:302] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f24b15e355d64db8b036a77d11ef213c; no voters:
I20260619 18:48:04.407670 23004 leader_election.cc:288] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:48:04.407778 23009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:48:04.408028 23009 raft_consensus.cc:686] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [term 1 LEADER]: Becoming Leader. State: Replica: f24b15e355d64db8b036a77d11ef213c, State: Running, Role: LEADER
I20260619 18:48:04.408378 23009 consensus_queue.cc:237] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f24b15e355d64db8b036a77d11ef213c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41455 } }
I20260619 18:48:04.408916 23004 sys_catalog.cc:564] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:48:04.410117 23011 sys_catalog.cc:455] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f24b15e355d64db8b036a77d11ef213c. Latest consensus state: current_term: 1 leader_uuid: "f24b15e355d64db8b036a77d11ef213c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f24b15e355d64db8b036a77d11ef213c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41455 } } }
I20260619 18:48:04.410113 23010 sys_catalog.cc:455] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f24b15e355d64db8b036a77d11ef213c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f24b15e355d64db8b036a77d11ef213c" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41455 } } }
I20260619 18:48:04.410300 23011 sys_catalog.cc:458] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [sys.catalog]: This master's current role is: LEADER
I20260619 18:48:04.410317 23010 sys_catalog.cc:458] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c [sys.catalog]: This master's current role is: LEADER
I20260619 18:48:04.410805 23018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:48:04.414358 23018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:48:04.420205 23018 catalog_manager.cc:1347] Generated new cluster ID: 68689c91f55745019abe26ed184e10f4
I20260619 18:48:04.420291 23018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:48:04.431897 23018 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:48:04.432261 23018 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:48:04.440828 23018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f24b15e355d64db8b036a77d11ef213c: Generated new TSK 0
I20260619 18:48:04.441742 23018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:48:04.448243 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41455
--builtin_ntp_servers=127.12.234.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 18:48:04.561719 23028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:04.562016 23028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:04.562057 23028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:04.566464 23028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:04.566524 23028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:04.566593 23028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:48:04.571552 23028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:04.572676 23028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:04.574152 23028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:04.583690 23036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:04.583690 23034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:04.583868 23028 server_base.cc:1034] running on GCE node
W20260619 18:48:04.583694 23033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:04.584403 23028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:04.585086 23028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:04.586252 23028 hybrid_clock.cc:648] HybridClock initialized: now 1781894884586222 us; error 48 us; skew 500 ppm
I20260619 18:48:04.588181 23028 webserver.cc:460] Webserver started at http://127.12.234.193:38043/ using document root <none> and password file <none>
I20260619 18:48:04.588752 23028 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:04.588819 23028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:04.589010 23028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:04.591100 23028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "baa51b8b260443da93f344559b5dc8f0"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "1ae105400482db34039dee66998744aa"
server_key_iv: "d6e778b49a62893d07934e784a2b4709"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.591639 23028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "baa51b8b260443da93f344559b5dc8f0"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "1ae105400482db34039dee66998744aa"
server_key_iv: "d6e778b49a62893d07934e784a2b4709"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.595242 23028 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:48:04.597859 23042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.599161 23028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:48:04.599272 23028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "baa51b8b260443da93f344559b5dc8f0"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "1ae105400482db34039dee66998744aa"
server_key_iv: "d6e778b49a62893d07934e784a2b4709"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.599402 23028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:04.603935 23028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:04.604548 23028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:04.604707 23028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:04.605237 23028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:04.606142 23028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:04.606249 23028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.606315 23028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:04.606346 23028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.614596 23028 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:38159
I20260619 18:48:04.614648 23155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:38159 every 8 connection(s)
I20260619 18:48:04.615501 23028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:48:04.621340 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:04.617211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:32805 (local address 127.12.234.254:41455)
0619 18:48:04.617364 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:04.617367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:04.618486 (+ 1119us) server_negotiation.cc:407] Connection header received
0619 18:48:04.619759 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:04.619763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:04.619806 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:04.619881 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:04.620913 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:04.620915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:04.620917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:48:04.620928 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:04.620947 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:48:04.620955 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:04.621033 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:04.621153 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 18:48:04.621195 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 18:48:04.622165 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:04.617624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41455 (local address 127.12.234.193:32805)
0619 18:48:04.618346 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:04.618387 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:04.619551 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:04.620043 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:04.620050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:04.620141 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:04.620636 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:48:04.620637 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:04.620677 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:48:04.620747 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:04.621025 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:04.621033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:04.621039 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:48:04.621189 (+ 150us) client_negotiation.cc:240] Negotiation successful
0619 18:48:04.621431 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":111,"threads_started":1}
I20260619 18:48:04.623438 23156 heartbeater.cc:347] Connected to a master server at 127.12.234.254:41455
I20260619 18:48:04.623641 23156 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:04.624028 23156 heartbeater.cc:511] Master 127.12.234.254:41455 requested a full tablet report, sending...
I20260619 18:48:04.624576 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 23028
I20260619 18:48:04.624670 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:48:04.624997 13227 external_mini_cluster.cc:1428] Setting key 30cb2f6a2ea8f11e29b7c44cb3ad6e80
I20260619 18:48:04.625665 22968 ts_manager.cc:194] Registered new tserver with Master: baa51b8b260443da93f344559b5dc8f0 (127.12.234.193:38159)
I20260619 18:48:04.627213 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41455
--builtin_ntp_servers=127.12.234.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 18:48:04.738430 23159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:04.738718 23159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:04.738757 23159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:04.742831 23159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:04.742899 23159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:04.742957 23159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:48:04.747890 23159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:04.749192 23159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:04.750766 23159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:04.760375 23167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:04.760365 23165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:04.760365 23164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:04.760684 23159 server_base.cc:1034] running on GCE node
I20260619 18:48:04.761059 23159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:04.761634 23159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:04.762786 23159 hybrid_clock.cc:648] HybridClock initialized: now 1781894884762768 us; error 30 us; skew 500 ppm
I20260619 18:48:04.764631 23159 webserver.cc:460] Webserver started at http://127.12.234.194:39675/ using document root <none> and password file <none>
I20260619 18:48:04.765185 23159 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:04.765251 23159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:04.765437 23159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:04.767529 23159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "8703dd02fbfc4f49b3baf03ead866c39"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "dc3f2e077c83cd79dc1c524400cf12c7"
server_key_iv: "37da11a034a7bd5dcb8f515222d5fc2e"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.768072 23159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "8703dd02fbfc4f49b3baf03ead866c39"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "dc3f2e077c83cd79dc1c524400cf12c7"
server_key_iv: "37da11a034a7bd5dcb8f515222d5fc2e"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.771809 23159 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 18:48:04.774392 23173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.775651 23159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:48:04.775750 23159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "8703dd02fbfc4f49b3baf03ead866c39"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "dc3f2e077c83cd79dc1c524400cf12c7"
server_key_iv: "37da11a034a7bd5dcb8f515222d5fc2e"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.775862 23159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:04.780556 23159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:04.781150 23159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:04.781311 23159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:04.781829 23159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:04.782732 23159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:04.782778 23159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.782814 23159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:04.782827 23159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.791422 23159 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:40501
I20260619 18:48:04.791447 23286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:40501 every 8 connection(s)
I20260619 18:48:04.792353 23159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:48:04.793241 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 23159
I20260619 18:48:04.793390 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:48:04.793695 13227 external_mini_cluster.cc:1428] Setting key f615042d56a9e753f636786e2ae538ed
I20260619 18:48:04.795873 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41455
--builtin_ntp_servers=127.12.234.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 18:48:04.798630 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:04.794343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:43067 (local address 127.12.234.254:41455)
0619 18:48:04.794476 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:04.794480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:04.795672 (+ 1192us) server_negotiation.cc:407] Connection header received
0619 18:48:04.796972 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:04.796976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:04.797021 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:04.797094 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:04.798163 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:04.798166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:04.798169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:48:04.798200 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:04.798222 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:48:04.798231 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:04.798315 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:04.798453 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 18:48:04.798498 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:48:04.799544 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:04.794813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41455 (local address 127.12.234.194:43067)
0619 18:48:04.795529 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:04.795571 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:04.796754 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:04.797258 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:04.797264 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:04.797365 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:04.797903 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:48:04.797904 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:04.797946 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:48:04.798018 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:04.798320 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:04.798327 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:04.798334 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 18:48:04.798499 (+ 165us) client_negotiation.cc:240] Negotiation successful
0619 18:48:04.798748 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":127,"threads_started":1}
I20260619 18:48:04.800657 23287 heartbeater.cc:347] Connected to a master server at 127.12.234.254:41455
I20260619 18:48:04.800887 23287 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:04.801277 23287 heartbeater.cc:511] Master 127.12.234.254:41455 requested a full tablet report, sending...
I20260619 18:48:04.802353 22968 ts_manager.cc:194] Registered new tserver with Master: 8703dd02fbfc4f49b3baf03ead866c39 (127.12.234.194:40501)
W20260619 18:48:04.904781 23290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:04.905055 23290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:04.905097 23290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:04.909188 23290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:04.909255 23290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:04.909314 23290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:48:04.914330 23290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:04.915401 23290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:04.916785 23290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:48:04.927179 23290 server_base.cc:1034] running on GCE node
W20260619 18:48:04.928694 23296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:04.928709 23298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:04.931641 23295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:04.932428 23290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:04.933243 23290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:04.934407 23290 hybrid_clock.cc:648] HybridClock initialized: now 1781894884934330 us; error 94 us; skew 500 ppm
I20260619 18:48:04.936862 23290 webserver.cc:460] Webserver started at http://127.12.234.195:40329/ using document root <none> and password file <none>
I20260619 18:48:04.937606 23290 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:04.937695 23290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:04.937947 23290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:04.940704 23290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "cf8f3808c86b42c2a1c03cfbdfc901a1"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "48b62c641e6823bda74a2fbed4b7af8e"
server_key_iv: "0e88f8f87af37626b841039e282a20b4"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.941430 23290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "cf8f3808c86b42c2a1c03cfbdfc901a1"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "48b62c641e6823bda74a2fbed4b7af8e"
server_key_iv: "0e88f8f87af37626b841039e282a20b4"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.946640 23290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 18:48:04.949608 23304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.950968 23290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:48:04.951081 23290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "cf8f3808c86b42c2a1c03cfbdfc901a1"
format_stamp: "Formatted at 2026-06-19 18:48:04 on dist-test-slave-fljv"
server_key: "48b62c641e6823bda74a2fbed4b7af8e"
server_key_iv: "0e88f8f87af37626b841039e282a20b4"
server_key_version: "encryptionkey@0"
I20260619 18:48:04.951193 23290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:04.955696 23290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:04.956314 23290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:04.956466 23290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:04.956981 23290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:04.957867 23290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:04.957912 23290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.957960 23290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:04.957988 23290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:04.966435 23290 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:33043
I20260619 18:48:04.966486 23417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:33043 every 8 connection(s)
I20260619 18:48:04.967391 23290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:48:04.971939 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 23290
I20260619 18:48:04.972045 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:48:04.972339 13227 external_mini_cluster.cc:1428] Setting key 629c064e344209978d600594fe9d85a4
I20260619 18:48:04.973800 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:04.969252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:37747 (local address 127.12.234.254:41455)
0619 18:48:04.969399 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:04.969405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:04.970549 (+ 1144us) server_negotiation.cc:407] Connection header received
0619 18:48:04.971886 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:04.971890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:04.971935 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:04.972014 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:04.973381 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:04.973383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:04.973385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:48:04.973396 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:04.973416 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:48:04.973424 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:04.973506 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:04.973639 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 18:48:04.973677 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:48:04.974833 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:04.969685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41455 (local address 127.12.234.195:37747)
0619 18:48:04.970404 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:04.970448 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:04.971620 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:04.972227 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:04.972237 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:04.972364 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:04.973081 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:48:04.973083 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:04.973145 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:48:04.973235 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:04.973505 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:04.973514 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:04.973519 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 18:48:04.973674 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 18:48:04.973912 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":103,"threads_started":1}
I20260619 18:48:04.975831 23418 heartbeater.cc:347] Connected to a master server at 127.12.234.254:41455
I20260619 18:48:04.976046 23418 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:04.976441 23418 heartbeater.cc:511] Master 127.12.234.254:41455 requested a full tablet report, sending...
I20260619 18:48:04.977416 22968 ts_manager.cc:194] Registered new tserver with Master: cf8f3808c86b42c2a1c03cfbdfc901a1 (127.12.234.195:33043)
I20260619 18:48:04.986445 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 18:48:04.989130 23007 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:48:04.988339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.12.234.254:41455)
0619 18:48:04.988472 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:04.988477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:04.988585 (+ 108us) server_negotiation.cc:407] Connection header received
0619 18:48:04.988691 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:04.988693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:04.988726 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:04.988789 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:04.988996 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48746: BlockingRecv error: recv got EOF from 127.0.0.1:48746 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:48:04.989238 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 23028
I20260619 18:48:04.993891 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 23159
I20260619 18:48:04.998409 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 23290
I20260619 18:48:05.002748 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 22937
2026-06-19T18:48:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (802 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:48:05 dist-test-slave-fljv krb5kdc[23428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:48:05 dist-test-slave-fljv krb5kdc[23428](info): set up 2 sockets
Jun 19 18:48:05 dist-test-slave-fljv krb5kdc[23428](info): commencing operation
krb5kdc: starting...
W20260619 18:48:05.974879 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:48:06 dist-test-slave-fljv krb5kdc[23428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:48:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:48:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:48:06.093670 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35521
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:46255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:35521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:48:06.206148 23441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:06.206480 23441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:06.206537 23441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:06.210671 23441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:48:06.210741 23441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:06.210767 23441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:06.210788 23441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:48:06.210815 23441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:48:06.216531 23441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:46255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:35521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:06.217715 23441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:06.218849 23441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:06.228363 23447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:06.228369 23446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:06.228471 23449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:06.228739 23441 server_base.cc:1034] running on GCE node
I20260619 18:48:06.229351 23441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:06.230598 23441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:06.231763 23441 hybrid_clock.cc:648] HybridClock initialized: now 1781894886231744 us; error 37 us; skew 500 ppm
Jun 19 18:48:06 dist-test-slave-fljv krb5kdc[23428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:48:06.234715 23441 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:48:06.235711 23441 webserver.cc:460] Webserver started at http://127.12.234.254:39509/ using document root <none> and password file <none>
I20260619 18:48:06.236284 23441 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:06.236356 23441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:06.236558 23441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:06.238672 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "dfc65ba593aa4efd9c9890ff48615cdc"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "f1ae154d47e55b5a804ea9af7ddb2a32"
server_key_iv: "65b48c9251061beff4cc3f07d6390bbd"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.239207 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "dfc65ba593aa4efd9c9890ff48615cdc"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "f1ae154d47e55b5a804ea9af7ddb2a32"
server_key_iv: "65b48c9251061beff4cc3f07d6390bbd"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.242977 23441 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:48:06.245642 23456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:06.247015 23441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:48:06.247139 23441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "dfc65ba593aa4efd9c9890ff48615cdc"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "f1ae154d47e55b5a804ea9af7ddb2a32"
server_key_iv: "65b48c9251061beff4cc3f07d6390bbd"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.247279 23441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:06.271342 23441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:06.274734 23441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:06.274927 23441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:06.281847 23441 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:35521
I20260619 18:48:06.282101 23508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:35521 every 8 connection(s)
I20260619 18:48:06.282887 23441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:48:06.285998 23509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:06.290264 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 23441
I20260619 18:48:06.290390 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:48:06.290678 13227 external_mini_cluster.cc:1428] Setting key db843f676dcf7170aa64838557f10018
I20260619 18:48:06.292392 23509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc: Bootstrap starting.
I20260619 18:48:06.295269 23509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:06.296043 23509 log.cc:824] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:06.298323 23509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc: No bootstrap required, opened a new log
Jun 19 18:48:06 dist-test-slave-fljv krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:48:06.302312 23509 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc65ba593aa4efd9c9890ff48615cdc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35521 } }
I20260619 18:48:06.302572 23509 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:06.302631 23509 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc65ba593aa4efd9c9890ff48615cdc, State: Initialized, Role: FOLLOWER
I20260619 18:48:06.303134 23509 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc65ba593aa4efd9c9890ff48615cdc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35521 } }
I20260619 18:48:06.303318 23509 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:48:06.303387 23509 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:48:06.303486 23509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:06.304678 23509 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc65ba593aa4efd9c9890ff48615cdc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35521 } }
I20260619 18:48:06.305022 23509 leader_election.cc:302] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfc65ba593aa4efd9c9890ff48615cdc; no voters:
I20260619 18:48:06.305335 23509 leader_election.cc:288] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:48:06.305444 23514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:48:06.305640 23514 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [term 1 LEADER]: Becoming Leader. State: Replica: dfc65ba593aa4efd9c9890ff48615cdc, State: Running, Role: LEADER
I20260619 18:48:06.305769 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:06.292155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.12.234.254:35521)
0619 18:48:06.292587 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:06.292606 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:06.292643 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:48:06.293306 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:06.293329 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:06.293685 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:06.293985 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:06.295185 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.296315 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.297355 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.297837 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.297971 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:06.300670 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:06.300676 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:06.300691 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:06.300722 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:06.302461 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:06.302975 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:06.302981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:06.302988 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:06.303142 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:06.303438 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:06.303442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:06.303444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:06.303963 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:06.304131 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:06.304423 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 18:48:06.304726 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":114,"threads_started":1}
I20260619 18:48:06.305955 23514 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc65ba593aa4efd9c9890ff48615cdc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35521 } }
I20260619 18:48:06.306794 23509 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:48:06.307183 23515 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfc65ba593aa4efd9c9890ff48615cdc. Latest consensus state: current_term: 1 leader_uuid: "dfc65ba593aa4efd9c9890ff48615cdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc65ba593aa4efd9c9890ff48615cdc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35521 } } }
I20260619 18:48:06.307289 23515 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [sys.catalog]: This master's current role is: LEADER
I20260619 18:48:06.307600 23516 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfc65ba593aa4efd9c9890ff48615cdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc65ba593aa4efd9c9890ff48615cdc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35521 } } }
I20260619 18:48:06.307669 23516 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc [sys.catalog]: This master's current role is: LEADER
I20260619 18:48:06.308115 23518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:48:06.311357 23518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:48:06.317278 23518 catalog_manager.cc:1347] Generated new cluster ID: ef4326991adb48f8b4190589786846e5
I20260619 18:48:06.317345 23518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:48:06.329954 23518 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:48:06.331456 23518 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:48:06.338423 23518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc: Generated new TSK 0
I20260619 18:48:06.339315 23518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:48:06.376993 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35521
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:46255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:48:06.491513 23535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:06.491811 23535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:06.491871 23535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:06.496019 23535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:06.496093 23535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:06.496182 23535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:48:06.501672 23535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:46255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:06.502902 23535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:06.503947 23535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:06.513831 23540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:06.513904 23535 server_base.cc:1034] running on GCE node
W20260619 18:48:06.513831 23543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:06.513834 23541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:06.514472 23535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:06.515156 23535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:06.516346 23535 hybrid_clock.cc:648] HybridClock initialized: now 1781894886516320 us; error 38 us; skew 500 ppm
Jun 19 18:48:06 dist-test-slave-fljv krb5kdc[23428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:48:06.519146 23535 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:48:06.520022 23535 webserver.cc:460] Webserver started at http://127.12.234.193:43087/ using document root <none> and password file <none>
I20260619 18:48:06.520558 23535 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:06.520623 23535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:06.520838 23535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:06.523057 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "0e1029196435451691bcde3259e2a683"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "3600a2154fa6c2f39cd7537ac69976c9"
server_key_iv: "1ff2628d58d642740fa1478962c43457"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.523659 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "0e1029196435451691bcde3259e2a683"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "3600a2154fa6c2f39cd7537ac69976c9"
server_key_iv: "1ff2628d58d642740fa1478962c43457"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.527740 23535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:48:06.530347 23550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:06.531649 23535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:48:06.531786 23535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "0e1029196435451691bcde3259e2a683"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "3600a2154fa6c2f39cd7537ac69976c9"
server_key_iv: "1ff2628d58d642740fa1478962c43457"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.531896 23535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:06.545583 23535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:06.548905 23535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:06.549086 23535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:06.549691 23535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:06.550567 23535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:06.550613 23535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:06.550674 23535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:06.550698 23535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:06.559319 23535 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:46185
I20260619 18:48:06.559351 23663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:46185 every 8 connection(s)
I20260619 18:48:06.560240 23535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:48:06.563905 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 23535
I20260619 18:48:06.564042 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:48:06.564360 13227 external_mini_cluster.cc:1428] Setting key 1c2a883f658ce8d9b6fd7950ecb35ce3
Jun 19 18:48:06 dist-test-slave-fljv krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:48:06.574756 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:06.562101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:34609 (local address 127.12.234.254:35521)
0619 18:48:06.562255 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:06.562259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:06.563169 (+ 910us) server_negotiation.cc:407] Connection header received
0619 18:48:06.564831 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:06.564835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:06.564896 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:06.564979 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:06.566412 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.566954 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.568183 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.568411 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.568497 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:06.571150 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:06.571153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:06.571155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:06.571181 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:06.572550 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:06.573108 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:06.573111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:06.573112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:06.573196 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:06.573513 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:06.573516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:06.573517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:06.573699 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:06.573813 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:06.574375 (+ 562us) server_negotiation.cc:299] Negotiation successful
0619 18:48:06.574510 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:48:06.575662 23666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:06.562550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35521 (local address 127.12.234.193:34609)
0619 18:48:06.563009 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:06.563053 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:06.564579 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:06.565190 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:06.565198 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:06.565562 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:06.566259 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:06.566286 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.567067 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.567071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:06.568058 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:06.568067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.568548 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.568552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:06.568720 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:06.569280 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:06.569303 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:06.570972 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:06.572671 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:06.572678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:06.572696 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:06.573009 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:06.573276 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:06.573279 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:06.573281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:06.573424 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:06.573824 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:06.573832 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:06.574100 (+ 268us) client_negotiation.cc:769] Sending connection context
0619 18:48:06.574314 (+ 214us) client_negotiation.cc:240] Negotiation successful
0619 18:48:06.574676 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
I20260619 18:48:06.576941 23664 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35521
I20260619 18:48:06.577178 23664 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:06.577690 23664 heartbeater.cc:511] Master 127.12.234.254:35521 requested a full tablet report, sending...
I20260619 18:48:06.579514 23473 ts_manager.cc:194] Registered new tserver with Master: 0e1029196435451691bcde3259e2a683 (127.12.234.193:46185)
I20260619 18:48:06.581133 23473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:34609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:48:06.593039 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35521
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:46255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:48:06.703501 23669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:06.703794 23669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:06.703837 23669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:06.708189 23669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:06.708273 23669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:06.708339 23669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:48:06.714269 23669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:46255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:06.715314 23669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:06.716303 23669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:06.726011 23677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:06.726013 23674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:06.726081 23669 server_base.cc:1034] running on GCE node
W20260619 18:48:06.726013 23675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:06.726632 23669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:06.727288 23669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:06.728453 23669 hybrid_clock.cc:648] HybridClock initialized: now 1781894886728433 us; error 33 us; skew 500 ppm
Jun 19 18:48:06 dist-test-slave-fljv krb5kdc[23428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:48:06.731174 23669 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:48:06.732069 23669 webserver.cc:460] Webserver started at http://127.12.234.194:37595/ using document root <none> and password file <none>
I20260619 18:48:06.732612 23669 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:06.732678 23669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:06.732879 23669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:06.735003 23669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "95e3201ab6e14593b9361dcda6d179dd"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "0cd12dee288153554c6c5189dc370feb"
server_key_iv: "ae007c5c2b3ea7c80d0b61a9548cb6c2"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.735536 23669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "95e3201ab6e14593b9361dcda6d179dd"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "0cd12dee288153554c6c5189dc370feb"
server_key_iv: "ae007c5c2b3ea7c80d0b61a9548cb6c2"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.739267 23669 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:48:06.741689 23684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:06.743041 23669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:48:06.743156 23669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "95e3201ab6e14593b9361dcda6d179dd"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "0cd12dee288153554c6c5189dc370feb"
server_key_iv: "ae007c5c2b3ea7c80d0b61a9548cb6c2"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.743260 23669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:06.771111 23669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:06.774271 23669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:06.774451 23669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:06.775050 23669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:06.775871 23669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:06.775916 23669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:06.775969 23669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:06.775997 23669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:06.784298 23669 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:37735
I20260619 18:48:06.784345 23797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:37735 every 8 connection(s)
I20260619 18:48:06.785209 23669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:48:06.789588 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 23669
I20260619 18:48:06.789737 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:48:06.790066 13227 external_mini_cluster.cc:1428] Setting key 26fb07c402ab797f66467ba3f61d25c1
Jun 19 18:48:06 dist-test-slave-fljv krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:48:06.799621 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:06.787192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:57935 (local address 127.12.234.254:35521)
0619 18:48:06.787333 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:06.787337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:06.788213 (+ 876us) server_negotiation.cc:407] Connection header received
0619 18:48:06.789717 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:06.789721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:06.789786 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:06.789867 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:06.791245 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.791763 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.792889 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.793110 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.793200 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:06.795933 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:06.795937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:06.795939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:06.795966 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:06.797317 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:06.797944 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:06.797946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:06.797948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:06.798019 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:06.798419 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:06.798421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:06.798422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:06.798606 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:06.798706 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:06.799383 (+ 677us) server_negotiation.cc:299] Negotiation successful
0619 18:48:06.799492 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 18:48:06.800550 23800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:06.787583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35521 (local address 127.12.234.194:57935)
0619 18:48:06.788044 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:06.788087 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:06.789451 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:06.790085 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:06.790096 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:06.790594 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:06.791096 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:06.791109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.791880 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.791883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:06.792773 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:06.792781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:06.793253 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:06.793257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:06.793420 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:06.793999 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:06.794017 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:06.795715 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:06.797436 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:06.797444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:06.797464 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:06.797847 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:06.798102 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:06.798106 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:06.798108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:06.798329 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:06.798721 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:06.798729 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:06.799097 (+ 368us) client_negotiation.cc:769] Sending connection context
0619 18:48:06.799314 (+ 217us) client_negotiation.cc:240] Negotiation successful
0619 18:48:06.799672 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
I20260619 18:48:06.801788 23798 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:48:06.802066 23798 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:06.802712 23798 heartbeater.cc:511] Master 127.12.234.254:35521 requested a full tablet report, sending...
I20260619 18:48:06.803862 23473 ts_manager.cc:194] Registered new tserver with Master: 95e3201ab6e14593b9361dcda6d179dd (127.12.234.194:37735)
I20260619 18:48:06.804430 23473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:57935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:48:06.817641 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35521
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:46255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:48:06.928365 23803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:06.928653 23803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:06.928716 23803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:06.932909 23803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:06.932977 23803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:06.933060 23803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:48:06.938700 23803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:46255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:06.939800 23803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:06.940845 23803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:06.950965 23809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:06.950930 23808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:06.950954 23811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:06.951237 23803 server_base.cc:1034] running on GCE node
I20260619 18:48:06.951805 23803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:06.952464 23803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:06.953624 23803 hybrid_clock.cc:648] HybridClock initialized: now 1781894886953599 us; error 36 us; skew 500 ppm
Jun 19 18:48:06 dist-test-slave-fljv krb5kdc[23428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:48:06.956636 23803 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:48:06.957619 23803 webserver.cc:460] Webserver started at http://127.12.234.195:44221/ using document root <none> and password file <none>
I20260619 18:48:06.958218 23803 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:06.958292 23803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:06.958514 23803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:48:06.960654 23803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "a2e76091af834891be28a8645dc09f1c"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "71da9a69d616dbc00a584eff50e9cc94"
server_key_iv: "911ff19789a653f055e9214a1bee9263"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.961233 23803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "a2e76091af834891be28a8645dc09f1c"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "71da9a69d616dbc00a584eff50e9cc94"
server_key_iv: "911ff19789a653f055e9214a1bee9263"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.965220 23803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 18:48:06.968021 23818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:06.969473 23803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:48:06.969614 23803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "a2e76091af834891be28a8645dc09f1c"
format_stamp: "Formatted at 2026-06-19 18:48:06 on dist-test-slave-fljv"
server_key: "71da9a69d616dbc00a584eff50e9cc94"
server_key_iv: "911ff19789a653f055e9214a1bee9263"
server_key_version: "encryptionkey@0"
I20260619 18:48:06.969726 23803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:06.994910 23803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:06.998517 23803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:06.998716 23803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:06.999332 23803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:07.000211 23803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:07.000260 23803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:07.000314 23803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:07.000344 23803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:07.009348 23803 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:45803
I20260619 18:48:07.009390 23931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:45803 every 8 connection(s)
I20260619 18:48:07.010349 23803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:48:07.013890 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 23803
I20260619 18:48:07.013993 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:48:07.014364 13227 external_mini_cluster.cc:1428] Setting key 5bf0b043fc3cf1ea207264d57ac3e6be
Jun 19 18:48:07 dist-test-slave-fljv krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894886, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:48:07.024796 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.012219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:33249 (local address 127.12.234.254:35521)
0619 18:48:07.012368 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:07.012372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:07.013343 (+ 971us) server_negotiation.cc:407] Connection header received
0619 18:48:07.014853 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:07.014857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:07.014913 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:07.014994 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:07.016627 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.017148 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.018337 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.018563 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.018641 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.021108 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:07.021112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:07.021114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:07.021138 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:07.022701 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:07.023199 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:07.023202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:07.023203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:07.023324 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:07.023727 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:07.023729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:07.023731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:07.023920 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:07.023996 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:07.024558 (+ 562us) server_negotiation.cc:299] Negotiation successful
0619 18:48:07.024667 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 18:48:07.025684 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.012682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35521 (local address 127.12.234.195:33249)
0619 18:48:07.013137 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:07.013232 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:07.014587 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:07.015282 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:07.015296 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:07.015782 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:07.016486 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.016507 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.017274 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.017276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.018172 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.018215 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.018684 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.018686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.018828 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.019406 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:07.019423 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:07.020943 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:07.022833 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:07.022839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:07.022854 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:07.023123 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:07.023434 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:07.023437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:07.023438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:07.023610 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:07.024031 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:07.024038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:07.024331 (+ 293us) client_negotiation.cc:769] Sending connection context
0619 18:48:07.024501 (+ 170us) client_negotiation.cc:240] Negotiation successful
0619 18:48:07.024788 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":109,"threads_started":1}
I20260619 18:48:07.027029 23932 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35521
I20260619 18:48:07.027374 23932 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:07.028015 23932 heartbeater.cc:511] Master 127.12.234.254:35521 requested a full tablet report, sending...
I20260619 18:48:07.029110 23473 ts_manager.cc:194] Registered new tserver with Master: a2e76091af834891be28a8645dc09f1c (127.12.234.195:45803)
I20260619 18:48:07.029673 23473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:33249
I20260619 18:48:07.040769 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:48:07.051784 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.042907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.12.234.254:35521)
0619 18:48:07.043039 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:07.043043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:07.043219 (+ 176us) server_negotiation.cc:407] Connection header received
0619 18:48:07.043280 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:07.043283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:07.043329 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:07.043403 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:07.044090 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.044706 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.045955 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.046428 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.046540 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.048159 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:07.048163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:07.048165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:07.048197 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:07.049805 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:07.050391 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:07.050394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:07.050395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:07.050493 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:07.050866 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:07.050868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:07.050869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:07.051181 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:07.051315 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:07.051534 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 18:48:07.051647 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 18:48:07.054999 23473 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:48:07.058252 23473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:48:07.072873 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.066892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:45803 (local address 127.0.0.1:53638)
0619 18:48:07.067985 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:07.067997 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:07.068082 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:07.068563 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:07.068566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:07.068580 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:07.068685 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.068690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.070157 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.070160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.071179 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.071188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.072232 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.072235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.072345 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.072394 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 18:48:07.072513 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 18:48:07.072696 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260619 18:48:07.072883 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.067958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.12.234.193:46185)
0619 18:48:07.068329 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:07.068347 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:07.068388 (+ 41us) server_negotiation.cc:407] Connection header received
0619 18:48:07.068542 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:07.068548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:07.068750 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:07.068897 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:07.069036 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.069751 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.071468 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.072221 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.072337 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.072436 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:07.072641 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 18:48:07.072740 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":112,"threads_started":1}
I20260619 18:48:07.072878 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.067220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53638 (local address 127.12.234.195:45803)
0619 18:48:07.067664 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:07.067677 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:07.068169 (+ 492us) server_negotiation.cc:407] Connection header received
0619 18:48:07.068259 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:07.068266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:07.068440 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:07.068593 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:07.068791 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.069986 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.071296 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.072121 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.072255 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.072408 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:07.072591 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 18:48:07.072696 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":133,"threads_started":1}
I20260619 18:48:07.073197 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.067550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:46185 (local address 127.0.0.1:37724)
0619 18:48:07.068276 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:07.068289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:07.068424 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:07.068869 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:07.068872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:07.068888 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:07.068963 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.068969 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.069880 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.069884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.071339 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.071350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.072364 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.072367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.072450 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.072469 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 18:48:07.072527 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 18:48:07.073095 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":488,"thread_start_us":80,"threads_started":1}
I20260619 18:48:07.074724 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.066777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:37735 (local address 127.0.0.1:53260)
0619 18:48:07.067435 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:07.067508 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:07.067642 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:07.068375 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:07.068381 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:07.068401 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:07.068553 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.068560 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.070118 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.070121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.071339 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.071350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.074325 (+ 2975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.074329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.074427 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.074456 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 18:48:07.074505 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 18:48:07.074584 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":46,"thread_start_us":93,"threads_started":1}
I20260619 18:48:07.075470 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.066943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53260 (local address 127.12.234.194:37735)
0619 18:48:07.067864 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:07.067875 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:07.067900 (+ 25us) server_negotiation.cc:407] Connection header received
0619 18:48:07.067977 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:07.067983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:07.068212 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:07.068686 (+ 474us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:07.068851 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.069986 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.072492 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.074154 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.075009 (+ 855us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.075155 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:07.075210 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 18:48:07.075320 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826,"thread_start_us":122,"threads_started":1}
I20260619 18:48:07.076300 23866 tablet_service.cc:1467] Processing CreateTablet for tablet 0c7788d46a7a4645a74f034d915791a3 (DEFAULT_TABLE table=test-table [id=29f34867984b4194b2b975d01b54fc0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:48:07.076273 23598 tablet_service.cc:1467] Processing CreateTablet for tablet 0c7788d46a7a4645a74f034d915791a3 (DEFAULT_TABLE table=test-table [id=29f34867984b4194b2b975d01b54fc0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:48:07.077306 23866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c7788d46a7a4645a74f034d915791a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:07.077309 23598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c7788d46a7a4645a74f034d915791a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:07.078510 23732 tablet_service.cc:1467] Processing CreateTablet for tablet 0c7788d46a7a4645a74f034d915791a3 (DEFAULT_TABLE table=test-table [id=29f34867984b4194b2b975d01b54fc0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:48:07.079619 23732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c7788d46a7a4645a74f034d915791a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:07.083784 23949 tablet_bootstrap.cc:492] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: Bootstrap starting.
I20260619 18:48:07.085513 23950 tablet_bootstrap.cc:492] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683: Bootstrap starting.
I20260619 18:48:07.086124 23949 tablet_bootstrap.cc:654] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:07.087368 23951 tablet_bootstrap.cc:492] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: Bootstrap starting.
I20260619 18:48:07.087375 23949 log.cc:824] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:07.088093 23950 tablet_bootstrap.cc:654] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:07.088994 23950 log.cc:824] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:07.090561 23951 tablet_bootstrap.cc:654] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:07.090739 23949 tablet_bootstrap.cc:492] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: No bootstrap required, opened a new log
I20260619 18:48:07.091002 23949 ts_tablet_manager.cc:1397] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 18:48:07.092406 23951 log.cc:824] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:07.095508 23951 tablet_bootstrap.cc:492] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: No bootstrap required, opened a new log
I20260619 18:48:07.095759 23951 ts_tablet_manager.cc:1397] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260619 18:48:07.097002 23949 raft_consensus.cc:348] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } }
I20260619 18:48:07.097536 23949 raft_consensus.cc:374] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:07.097645 23949 raft_consensus.cc:729] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e76091af834891be28a8645dc09f1c, State: Initialized, Role: FOLLOWER
I20260619 18:48:07.098220 23949 consensus_queue.cc:260] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } }
I20260619 18:48:07.098999 23932 heartbeater.cc:503] Master 127.12.234.254:35521 was elected leader, sending a full tablet report...
I20260619 18:48:07.099427 23949 ts_tablet_manager.cc:1428] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260619 18:48:07.100826 23951 raft_consensus.cc:348] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } }
I20260619 18:48:07.101118 23951 raft_consensus.cc:374] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:07.101178 23951 raft_consensus.cc:729] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95e3201ab6e14593b9361dcda6d179dd, State: Initialized, Role: FOLLOWER
I20260619 18:48:07.101717 23951 consensus_queue.cc:260] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } }
I20260619 18:48:07.102063 23950 tablet_bootstrap.cc:492] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683: No bootstrap required, opened a new log
I20260619 18:48:07.102347 23950 ts_tablet_manager.cc:1397] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.002s
I20260619 18:48:07.103009 23951 ts_tablet_manager.cc:1428] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 18:48:07.103387 23798 heartbeater.cc:503] Master 127.12.234.254:35521 was elected leader, sending a full tablet report...
I20260619 18:48:07.108047 23950 raft_consensus.cc:348] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } }
I20260619 18:48:07.108455 23950 raft_consensus.cc:374] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:07.108527 23950 raft_consensus.cc:729] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1029196435451691bcde3259e2a683, State: Initialized, Role: FOLLOWER
I20260619 18:48:07.109041 23950 consensus_queue.cc:260] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } }
I20260619 18:48:07.109788 23664 heartbeater.cc:503] Master 127.12.234.254:35521 was elected leader, sending a full tablet report...
I20260619 18:48:07.110065 23950 ts_tablet_manager.cc:1428] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260619 18:48:07.185173 23956 raft_consensus.cc:482] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:48:07.185425 23956 raft_consensus.cc:504] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } }
I20260619 18:48:07.186628 23956 leader_election.cc:288] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2e76091af834891be28a8645dc09f1c (127.12.234.195:45803), 0e1029196435451691bcde3259e2a683 (127.12.234.193:46185)
I20260619 18:48:07.191399 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.187442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:33699 (local address 127.12.234.193:46185)
0619 18:48:07.187552 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:07.187556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:07.187700 (+ 144us) server_negotiation.cc:407] Connection header received
0619 18:48:07.187808 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:07.187810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:07.187853 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:07.187919 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:07.188191 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.188833 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.190156 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.190789 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.190912 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.190966 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:07.191189 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 18:48:07.191265 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 18:48:07.191399 23800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.186935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:45803 (local address 127.12.234.194:54315)
0619 18:48:07.187058 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:07.187074 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:07.187171 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:07.187606 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:07.187610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:07.187633 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:07.187762 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.187770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.188729 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.188734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.190069 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.190082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.191022 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.191025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.191108 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.191125 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 18:48:07.191187 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 18:48:07.191271 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":1}
I20260619 18:48:07.191763 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.187052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:54315 (local address 127.12.234.195:45803)
0619 18:48:07.187224 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:07.187229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:07.187250 (+ 21us) server_negotiation.cc:407] Connection header received
0619 18:48:07.187317 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:07.187321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:07.187376 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:07.187472 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:07.187885 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.188607 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.190235 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.190829 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.190925 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.190966 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:07.191563 (+ 597us) server_negotiation.cc:299] Negotiation successful
0619 18:48:07.191647 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 18:48:07.191908 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.187210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:46185 (local address 127.12.234.194:33699)
0619 18:48:07.187612 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:07.187624 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:07.187716 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:07.187929 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:07.187932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:07.187948 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:07.188068 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.188075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.188949 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.188952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.190040 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:07.190050 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.190917 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.190921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:07.191035 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.191068 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 18:48:07.191165 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 18:48:07.191254 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":24,"thread_start_us":126,"threads_started":1}
I20260619 18:48:07.192413 23886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c7788d46a7a4645a74f034d915791a3" candidate_uuid: "95e3201ab6e14593b9361dcda6d179dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2e76091af834891be28a8645dc09f1c" is_pre_election: true
I20260619 18:48:07.192467 23618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c7788d46a7a4645a74f034d915791a3" candidate_uuid: "95e3201ab6e14593b9361dcda6d179dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1029196435451691bcde3259e2a683" is_pre_election: true
I20260619 18:48:07.192759 23618 raft_consensus.cc:2454] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95e3201ab6e14593b9361dcda6d179dd in term 0.
I20260619 18:48:07.192761 23886 raft_consensus.cc:2454] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95e3201ab6e14593b9361dcda6d179dd in term 0.
I20260619 18:48:07.193217 23687 leader_election.cc:302] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95e3201ab6e14593b9361dcda6d179dd, a2e76091af834891be28a8645dc09f1c; no voters:
I20260619 18:48:07.193512 23956 raft_consensus.cc:2781] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:48:07.193621 23956 raft_consensus.cc:482] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:48:07.193675 23956 raft_consensus.cc:3037] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:07.195086 23956 raft_consensus.cc:504] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } }
I20260619 18:48:07.195525 23956 leader_election.cc:288] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [CANDIDATE]: Term 1 election: Requested vote from peers a2e76091af834891be28a8645dc09f1c (127.12.234.195:45803), 0e1029196435451691bcde3259e2a683 (127.12.234.193:46185)
I20260619 18:48:07.195932 23886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c7788d46a7a4645a74f034d915791a3" candidate_uuid: "95e3201ab6e14593b9361dcda6d179dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2e76091af834891be28a8645dc09f1c"
I20260619 18:48:07.196055 23886 raft_consensus.cc:3037] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:07.196035 23618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c7788d46a7a4645a74f034d915791a3" candidate_uuid: "95e3201ab6e14593b9361dcda6d179dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1029196435451691bcde3259e2a683"
I20260619 18:48:07.196179 23618 raft_consensus.cc:3037] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:07.197152 23886 raft_consensus.cc:2454] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95e3201ab6e14593b9361dcda6d179dd in term 1.
I20260619 18:48:07.197419 23687 leader_election.cc:302] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95e3201ab6e14593b9361dcda6d179dd, a2e76091af834891be28a8645dc09f1c; no voters:
I20260619 18:48:07.197592 23956 raft_consensus.cc:2781] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:48:07.197608 23618 raft_consensus.cc:2454] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95e3201ab6e14593b9361dcda6d179dd in term 1.
I20260619 18:48:07.197832 23956 raft_consensus.cc:686] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 1 LEADER]: Becoming Leader. State: Replica: 95e3201ab6e14593b9361dcda6d179dd, State: Running, Role: LEADER
I20260619 18:48:07.198247 23956 consensus_queue.cc:237] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } }
I20260619 18:48:07.201632 23472 catalog_manager.cc:5495] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 95e3201ab6e14593b9361dcda6d179dd (127.12.234.194). New cstate: current_term: 1 leader_uuid: "95e3201ab6e14593b9361dcda6d179dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95e3201ab6e14593b9361dcda6d179dd" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 } health_report { overall_health: UNKNOWN } } }
I20260619 18:48:07.228915 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.224131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53262 (local address 127.12.234.194:37735)
0619 18:48:07.224274 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:07.224278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:07.224289 (+ 11us) server_negotiation.cc:407] Connection header received
0619 18:48:07.224370 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:07.224373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:07.224424 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:07.224517 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:48:07.224845 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.225661 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.227026 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:07.227514 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:07.227658 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:07.228074 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:48:07.228507 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:48:07.228603 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:07.228706 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 18:48:07.228763 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 18:48:07.235371 23886 raft_consensus.cc:1261] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [term 1 FOLLOWER]: Refusing update from remote peer 95e3201ab6e14593b9361dcda6d179dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:48:07.235371 23618 raft_consensus.cc:1261] T 0c7788d46a7a4645a74f034d915791a3 P 0e1029196435451691bcde3259e2a683 [term 1 FOLLOWER]: Refusing update from remote peer 95e3201ab6e14593b9361dcda6d179dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:48:07.236181 23959 consensus_queue.cc:1035] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e1029196435451691bcde3259e2a683" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:48:07.236378 23956 consensus_queue.cc:1035] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2e76091af834891be28a8645dc09f1c" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:48:07.246413 23962 mvcc.cc:205] Tried to move back new op lower bound from 7298641458107912192 to 7298641457966694400. Current Snapshot: MvccSnapshot[applied={T|T < 7298641458107912192}]
I20260619 18:48:07.247980 23964 mvcc.cc:205] Tried to move back new op lower bound from 7298641458107912192 to 7298641457966694400. Current Snapshot: MvccSnapshot[applied={T|T < 7298641458107912192}]
I20260619 18:48:07.254356 23963 mvcc.cc:205] Tried to move back new op lower bound from 7298641458107912192 to 7298641457966694400. Current Snapshot: MvccSnapshot[applied={T|T < 7298641458107912192 or (T in {7298641458107912192})}]
I20260619 18:48:07.256222 23473 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:48:07.256384 23473 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:48:07.258720 23473 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfc65ba593aa4efd9c9890ff48615cdc: Sending DeleteTablet for 3 replicas of tablet 0c7788d46a7a4645a74f034d915791a3
I20260619 18:48:07.259563 23732 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c7788d46a7a4645a74f034d915791a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:48:07 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:53260
I20260619 18:48:07.259650 23866 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c7788d46a7a4645a74f034d915791a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:48:07 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:53638
I20260619 18:48:07.259779 23598 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c7788d46a7a4645a74f034d915791a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:48:07 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:37724
I20260619 18:48:07.260691 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 23535
I20260619 18:48:07.263150 23971 tablet_replica.cc:330] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: stopping tablet replica
I20260619 18:48:07.263553 23971 raft_consensus.cc:2229] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:48:07.264113 23971 raft_consensus.cc:2258] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:48:07.265396 23971 ts_tablet_manager.cc:1905] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:48:07.266425 23972 tablet_replica.cc:330] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: stopping tablet replica
I20260619 18:48:07.266840 23972 raft_consensus.cc:2229] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:48:07.267305 23972 raft_consensus.cc:2258] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:48:07.268174 23971 ts_tablet_manager.cc:1918] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:48:07.268276 23971 log.cc:1196] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/0c7788d46a7a4645a74f034d915791a3
I20260619 18:48:07.268573 23971 ts_tablet_manager.cc:1939] T 0c7788d46a7a4645a74f034d915791a3 P 95e3201ab6e14593b9361dcda6d179dd: Deleting consensus metadata
I20260619 18:48:07.269011 23972 ts_tablet_manager.cc:1905] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 18:48:07.269238 23459 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:46185 (error 108)
I20260619 18:48:07.270033 23460 catalog_manager.cc:4841] TS 95e3201ab6e14593b9361dcda6d179dd (127.12.234.194:37735): tablet 0c7788d46a7a4645a74f034d915791a3 (table test-table [id=29f34867984b4194b2b975d01b54fc0b]) successfully deleted
I20260619 18:48:07.270357 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 23669
I20260619 18:48:07.272050 23972 ts_tablet_manager.cc:1918] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:48:07.272140 23972 log.cc:1196] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/0c7788d46a7a4645a74f034d915791a3
I20260619 18:48:07.272411 23972 ts_tablet_manager.cc:1939] T 0c7788d46a7a4645a74f034d915791a3 P a2e76091af834891be28a8645dc09f1c: Deleting consensus metadata
I20260619 18:48:07.273169 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:07.272818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:46185 (local address 127.0.0.1:37736)
0619 18:48:07.272967 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:07.273049 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 18:48:07.273327 23459 catalog_manager.cc:4584] TS 0e1029196435451691bcde3259e2a683 (127.12.234.193:46185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c7788d46a7a4645a74f034d915791a3: Network error: Client connection negotiation failed: client connection to 127.12.234.193:46185: connect: Connection refused (error 111)
I20260619 18:48:07.273679 23459 catalog_manager.cc:4841] TS a2e76091af834891be28a8645dc09f1c (127.12.234.195:45803): tablet 0c7788d46a7a4645a74f034d915791a3 (table test-table [id=29f34867984b4194b2b975d01b54fc0b]) successfully deleted
I20260619 18:48:07.276363 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 23803
I20260619 18:48:07.281838 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 23441
2026-06-19T18:48:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2280 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:48:07 dist-test-slave-fljv krb5kdc[23977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:48:07 dist-test-slave-fljv krb5kdc[23977](info): set up 2 sockets
Jun 19 18:48:07 dist-test-slave-fljv krb5kdc[23977](info): commencing operation
krb5kdc: starting...
W20260619 18:48:08.251783 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:48:08 dist-test-slave-fljv krb5kdc[23977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:48:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:48:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.234.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.234.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/rangeradmin_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/rangeradmin_127.12.234.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.234.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.234.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/rangerlookup_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/rangerlookup_127.12.234.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.234.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.234.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/HTTP_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/HTTP_127.12.234.212@KRBTEST.COM.keytab.
I20260619 18:48:08.433624 13227 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJa0qyg/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/postgres -l logfile start
2026-06-19 18:48:10.964 UTC [24007] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 18:48:10.964 UTC [24007] LOG: listening on IPv4 address "127.12.234.212", port 45213
2026-06-19 18:48:10.968 UTC [24007] LOG: listening on Unix socket "/tmp/.s.PGSQL.45213"
2026-06-19 18:48:10.988 UTC [24010] LOG: database system was shut down at 2026-06-19 18:48:09 UTC
2026-06-19 18:48:10.991 UTC [24007] LOG: database system is ready to accept connections
I20260619 18:48:11.862900 13227 mini_postgres.cc:96] Postgres bound to 45213
2026-06-19 18:48:11.869 UTC [24018] FATAL: database "slave" does not exist
127.12.234.212:45213 - accepting connections
I20260619 18:48:11.870002 13227 mini_ranger.cc:161] Starting Ranger...
I20260619 18:48:11.885270 13227 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 18:48:12.398653 13227 mini_ranger.cc:87] Created ranger Postgres database
I20260619 18:48:12.398747 13227 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-admin
2026-06-19 18:48:12,684 [I] DB FLAVOR :POSTGRES
2026-06-19 18:48:12,684 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 18:48:12,684 [I] Checking connection..
2026-06-19 18:48:12,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 18:48:12,948 [I] Checking connection passed.
2026-06-19 18:48:12,948 [I] --------- Verifying version history table ---------
2026-06-19 18:48:12,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 18:48:13,191 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 18:48:13,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 18:48:13,424 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 18:48:13,424 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 18:48:13,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 18:48:13.664 UTC [24082] WARNING: there is no transaction in progress
2026-06-19 18:48:13,669 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 18:48:13,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 18:48:13,931 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 18:48:13,931 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 18:48:13,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 18:48:14,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 18:48:14,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fljv.c.gcp-upstream.internal','N') ;"
2026-06-19 18:48:14,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 18:48:14,897 [I] Table x_portal_user does not exist in database ranger
2026-06-19 18:48:14,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 18:48:15,135 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 18:48:15,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 18:48:15,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 18:48:15,615 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 18:48:15,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 18:48:16.627 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.643 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.660 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.676 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.693 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.712 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.778 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.785 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.800 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.810 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.821 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.835 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.845 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.851 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.859 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:16.866 UTC [24190] WARNING: there is no transaction in progress
2026-06-19 18:48:17,526 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 18:48:17,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 18:48:17,781 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 18:48:17,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 18:48:18,015 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 18:48:18,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 18:48:18,254 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 18:48:18,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 18:48:18,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 18:48:18,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fljv.c.gcp-upstream.internal';"
2026-06-19 18:48:18,958 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 18:48:18,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 18:48:19,189 [I] DB_PATCHES have already been applied
I20260619 18:48:19.195492 13227 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJa0qyg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 18:48:19.195562 13227 mini_ranger.cc:193] Using host: 127.12.234.212
I20260619 18:48:19.198853 13227 mini_ranger.cc:227] Ranger admin URL: http://127.12.234.212:38543
Jun 19, 2026 6:48:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:48:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:48:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 6:48:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 6:48:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 6:48:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 6:48:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 18:48:20.192132 13227 mini_ranger.cc:160] Time spent starting Ranger: real 8.322s user 0.000s sys 0.010s
Jun 19, 2026 6:48:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 6:48:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 6:48:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 6:48:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 6:48:41 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 18:48:42.025333 13227 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.234.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.234.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/rangerkms_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/rangerkms_127.12.234.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.234.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.234.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.234.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/HTTP_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.234.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/HTTP_127.12.234.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 18:48:42 dist-test-slave-fljv krb5kdc[23977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894922, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 18:48:42.109230 13227 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 18:48:42.118639 13227 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 18:48:43.076108 13227 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 18:48:43.076193 13227 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-kms
I20260619 18:48:43.076287 13227 mini_ranger_kms.cc:227] Using postgres at 127.12.234.212:45213
2026-06-19 18:48:43,241 [I] DB FLAVOR :POSTGRES
2026-06-19 18:48:43,241 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 18:48:43,242 [I] Checking connection
2026-06-19 18:48:43,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 18:48:43,511 [I] connection success
2026-06-19 18:48:43,511 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 18:48:43,511 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 18:48:43,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 18:48:43,776 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 18:48:43,776 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 18:48:43,776 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 18:48:43,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:45213/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 18:48:44,054 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 18:48:44.520237 13227 mini_ranger_kms.cc:310] Created kms service
I20260619 18:48:44.721604 13227 mini_ranger_kms.cc:326] Created kudu user
I20260619 18:48:44.771862 13227 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 18:48:44 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894922, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:48:45.144434 13227 mini_ranger_kms.cc:384] Added ranger policy
I20260619 18:48:45.144585 13227 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 18:48:45.144606 13227 mini_ranger_kms.cc:240] Using host: 127.12.234.212
I20260619 18:48:45.148113 13227 mini_ranger_kms.cc:276] Ranger KMS PID: 24370
I20260619 18:48:45.148257 13227 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.234.212:55251
Jun 19, 2026 6:48:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:48:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:48:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 6:48:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 6:48:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 6:48:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 18:48:47.220001 13227 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.111s user 0.012s sys 0.137s
I20260619 18:48:47.220117 13227 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 18:48:47.220229 13227 mini_ranger_kms.cc:401] 127.12.234.212:55251/kms/v1/keys
Jun 19 18:48:47 dist-test-slave-fljv krb5kdc[23977](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781894927, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 18:48:48 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894927, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.212@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
Jun 19 18:48:48 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894927, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.212@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
Jun 19 18:48:50 dist-test-slave-fljv krb5kdc[23977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:48:50.697676 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:37695
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:37695
--ranger_config_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:55251/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:48:50.832522 24421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:50.832830 24421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:50.832885 24421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:50.837033 24421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:48:50.837111 24421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:50.837139 24421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:50.837158 24421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:48:50.837180 24421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:48:50.842939 24421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.234.212:55251/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:37695
--ranger_config_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:37695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:50.844089 24421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:50.845153 24421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:50.855581 24427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:50.855574 24429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:50.855654 24421 server_base.cc:1034] running on GCE node
W20260619 18:48:50.855669 24426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:50.856503 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:50.857740 24421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:50.858914 24421 hybrid_clock.cc:648] HybridClock initialized: now 1781894930858899 us; error 34 us; skew 500 ppm
Jun 19 18:48:50 dist-test-slave-fljv krb5kdc[23977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894930, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:48:50.865811 24421 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:48:50.866863 24421 webserver.cc:460] Webserver started at http://127.12.234.254:34213/ using document root <none> and password file <none>
I20260619 18:48:50.867419 24421 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:50.867476 24421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:50.867683 24421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 18:48:50 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894930, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:48:50.923389 24421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "deb95da8a0024736b4f9079e9110a40f"
format_stamp: "Formatted at 2026-06-19 18:48:50 on dist-test-slave-fljv"
server_key: "24b5dbfb084f8265b9d4a7853b45d4ba"
server_key_iv: "7eb7a0a46db1fdb3cb87b90446581cff"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:50.924081 24421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "deb95da8a0024736b4f9079e9110a40f"
format_stamp: "Formatted at 2026-06-19 18:48:50 on dist-test-slave-fljv"
server_key: "24b5dbfb084f8265b9d4a7853b45d4ba"
server_key_iv: "7eb7a0a46db1fdb3cb87b90446581cff"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:50.928073 24421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:48:50.942482 24436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:50.943939 24421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:48:50.944074 24421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "deb95da8a0024736b4f9079e9110a40f"
format_stamp: "Formatted at 2026-06-19 18:48:50 on dist-test-slave-fljv"
server_key: "24b5dbfb084f8265b9d4a7853b45d4ba"
server_key_iv: "7eb7a0a46db1fdb3cb87b90446581cff"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:50.944213 24421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:50.973244 24421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:50.976305 24421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:50.976492 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:50.984159 24421 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:37695
I20260619 18:48:50.984186 24488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:37695 every 8 connection(s)
I20260619 18:48:50.985314 24421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:48:50.986663 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 24421
I20260619 18:48:50.986797 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:48:50.987159 13227 external_mini_cluster.cc:1428] Setting key 0e9ff1d12265a84f93fe8daf116ffe90
I20260619 18:48:50.989017 24489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:50.995925 24489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f: Bootstrap starting.
Jun 19 18:48:50 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:48:50.998878 24489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:50.999747 24489 log.cc:824] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:51.002305 24489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f: No bootstrap required, opened a new log
I20260619 18:48:51.005587 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:50.988714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47420 (local address 127.12.234.254:37695)
0619 18:48:50.989234 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:50.989254 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:50.989299 (+ 45us) server_negotiation.cc:407] Connection header received
0619 18:48:50.990169 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:50.990229 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:50.990767 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:50.991234 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:50.992185 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:50.993877 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:50.995356 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:50.996110 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:50.996276 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:50.999499 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:50.999507 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:50.999529 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:50.999569 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:51.002058 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:51.002675 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:51.002681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:51.002689 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:51.002835 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:51.003152 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:51.003155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:51.003157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:51.003745 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:51.003950 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:51.004277 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 18:48:51.004577 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":159,"threads_started":1}
I20260619 18:48:51.006289 24489 raft_consensus.cc:348] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb95da8a0024736b4f9079e9110a40f" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37695 } }
I20260619 18:48:51.006577 24489 raft_consensus.cc:374] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:51.006672 24489 raft_consensus.cc:729] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deb95da8a0024736b4f9079e9110a40f, State: Initialized, Role: FOLLOWER
I20260619 18:48:51.007253 24489 consensus_queue.cc:260] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb95da8a0024736b4f9079e9110a40f" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37695 } }
I20260619 18:48:51.007413 24489 raft_consensus.cc:388] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:48:51.007524 24489 raft_consensus.cc:482] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:48:51.007673 24489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:51.008827 24489 raft_consensus.cc:504] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb95da8a0024736b4f9079e9110a40f" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37695 } }
I20260619 18:48:51.009172 24489 leader_election.cc:302] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: deb95da8a0024736b4f9079e9110a40f; no voters:
I20260619 18:48:51.009485 24489 leader_election.cc:288] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:48:51.009581 24494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:48:51.009747 24494 raft_consensus.cc:686] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [term 1 LEADER]: Becoming Leader. State: Replica: deb95da8a0024736b4f9079e9110a40f, State: Running, Role: LEADER
I20260619 18:48:51.010672 24494 consensus_queue.cc:237] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb95da8a0024736b4f9079e9110a40f" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37695 } }
I20260619 18:48:51.010905 24489 sys_catalog.cc:564] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:48:51.011997 24494 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [sys.catalog]: SysCatalogTable state changed. Reason: New leader deb95da8a0024736b4f9079e9110a40f. Latest consensus state: current_term: 1 leader_uuid: "deb95da8a0024736b4f9079e9110a40f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb95da8a0024736b4f9079e9110a40f" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37695 } } }
I20260619 18:48:51.012207 24494 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [sys.catalog]: This master's current role is: LEADER
I20260619 18:48:51.012674 24495 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "deb95da8a0024736b4f9079e9110a40f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb95da8a0024736b4f9079e9110a40f" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37695 } } }
I20260619 18:48:51.012763 24495 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f [sys.catalog]: This master's current role is: LEADER
I20260619 18:48:51.013111 24503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:48:51.016372 24503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:48:51.016803 24489 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 18:48:51.028287 24503 catalog_manager.cc:1347] Generated new cluster ID: 8e017b6cce0d4c5c894ff19c17f80b4b
I20260619 18:48:51.028458 24503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:48:51.040223 24503 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:48:51.041684 24503 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:48:51.048904 24503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f: Generated new TSK 0
I20260619 18:48:51.050530 24503 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:48:51.823273 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:55251/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37695
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:48:51.952924 24534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:51.953233 24534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:51.953279 24534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:51.957502 24534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:51.957572 24534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:51.957641 24534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:48:51.963168 24534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.234.212:55251/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:51.964308 24534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:51.965466 24534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:48:51.976009 24539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:51.976164 24540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:51.976009 24542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:51.976631 24534 server_base.cc:1034] running on GCE node
I20260619 18:48:51.977125 24534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:51.977983 24534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:51.979174 24534 hybrid_clock.cc:648] HybridClock initialized: now 1781894931979137 us; error 55 us; skew 500 ppm
Jun 19 18:48:51 dist-test-slave-fljv krb5kdc[23977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894931, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:48:51.983035 24534 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:48:51.984248 24534 webserver.cc:460] Webserver started at http://127.12.234.193:36431/ using document root <none> and password file <none>
I20260619 18:48:51.984928 24534 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:51.985071 24534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:51.985383 24534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 18:48:51 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894931, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:48:52.005044 24534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "2a14acc65f6249a2835450849a9d3fb5"
format_stamp: "Formatted at 2026-06-19 18:48:51 on dist-test-slave-fljv"
server_key: "3cbb616245116a5d5ee0f938d9be75e6"
server_key_iv: "10e5fd21a1bd9db391b8b900b72403dc"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:52.005978 24534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "2a14acc65f6249a2835450849a9d3fb5"
format_stamp: "Formatted at 2026-06-19 18:48:51 on dist-test-slave-fljv"
server_key: "3cbb616245116a5d5ee0f938d9be75e6"
server_key_iv: "10e5fd21a1bd9db391b8b900b72403dc"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:52.019860 24534 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.004s sys 0.000s
I20260619 18:48:52.038130 24551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:52.040077 24534 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:48:52.040308 24534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "2a14acc65f6249a2835450849a9d3fb5"
format_stamp: "Formatted at 2026-06-19 18:48:51 on dist-test-slave-fljv"
server_key: "3cbb616245116a5d5ee0f938d9be75e6"
server_key_iv: "10e5fd21a1bd9db391b8b900b72403dc"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:52.040555 24534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:52.077811 24534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:52.093577 24534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:52.093775 24534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:52.094511 24534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:52.095633 24534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:52.095685 24534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:52.095737 24534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:52.095758 24534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:52.107214 24534 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:46795
I20260619 18:48:52.108464 24534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:48:52.110050 24666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:46795 every 8 connection(s)
I20260619 18:48:52.112221 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 24534
I20260619 18:48:52.112342 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:48:52.112640 13227 external_mini_cluster.cc:1428] Setting key 16914b486f3b407774cad312f3945fcc
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
Jun 19 18:48:52 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894931, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:48:52.140673 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.115434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:59589 (local address 127.12.234.254:37695)
0619 18:48:52.120507 (+ 5073us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:52.120511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:52.120527 (+ 16us) server_negotiation.cc:407] Connection header received
0619 18:48:52.120570 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:52.120572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:52.120624 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:52.120717 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:52.122424 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.123308 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.125070 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.125427 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.125536 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.136094 (+ 10558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:52.136100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:52.136103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:52.136137 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:52.137972 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:52.138708 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:52.138712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:52.138714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:52.138810 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:52.139247 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:52.139251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:52.139253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:52.139508 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:52.139593 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:52.140334 (+ 741us) server_negotiation.cc:299] Negotiation successful
0619 18:48:52.140483 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4990,"thread_start_us":65,"threads_started":1}
I20260619 18:48:52.142760 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.114333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:37695 (local address 127.12.234.193:59589)
0619 18:48:52.118835 (+ 4502us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:52.118901 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:52.119990 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:52.121074 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:52.121085 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:52.121472 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:52.122243 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.122265 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.123460 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.123464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.124920 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.124931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.125737 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.125741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.125953 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.133905 (+ 7952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:52.133932 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:52.135793 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:52.138134 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:52.138143 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:52.138164 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:52.138580 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:52.138921 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:52.138924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:52.138926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:52.139129 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:52.139662 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:52.139670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:52.140039 (+ 369us) client_negotiation.cc:769] Sending connection context
0619 18:48:52.141222 (+ 1183us) client_negotiation.cc:240] Negotiation successful
0619 18:48:52.141612 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4243,"thread_start_us":143,"threads_started":1}
I20260619 18:48:52.144444 24667 heartbeater.cc:347] Connected to a master server at 127.12.234.254:37695
I20260619 18:48:52.144703 24667 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:52.145292 24667 heartbeater.cc:511] Master 127.12.234.254:37695 requested a full tablet report, sending...
I20260619 18:48:52.147246 24453 ts_manager.cc:194] Registered new tserver with Master: 2a14acc65f6249a2835450849a9d3fb5 (127.12.234.193:46795)
I20260619 18:48:52.148906 24453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:59589
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:48:52.170564 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:55251/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37695
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 18:48:52 dist-test-slave-fljv krb5kdc[23977](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781894932, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 18:48:52.336465 24673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:52.337357 24673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:52.337503 24673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:52.343842 24673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:52.344064 24673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:52.344220 24673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:48:52.352631 24673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.234.212:55251/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:52.354645 24673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:52.356166 24673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:48:52.370530 24673 server_base.cc:1034] running on GCE node
W20260619 18:48:52.370769 24682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:52.373863 24679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:52.374295 24680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:52.374795 24673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:52.375725 24673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:52.377003 24673 hybrid_clock.cc:648] HybridClock initialized: now 1781894932376921 us; error 102 us; skew 500 ppm
Jun 19 18:48:52 dist-test-slave-fljv krb5kdc[23977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894932, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:48:52.380674 24673 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:48:52.381876 24673 webserver.cc:460] Webserver started at http://127.12.234.194:37549/ using document root <none> and password file <none>
I20260619 18:48:52.382694 24673 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:52.382833 24673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:52.383116 24673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 18:48:52 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894932, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:48:52.410738 24673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "5d5afd3b8c7247be81ecafe4fd7562a0"
format_stamp: "Formatted at 2026-06-19 18:48:52 on dist-test-slave-fljv"
server_key: "73fa8e2d88135b2e8c0780a38b87f3fa"
server_key_iv: "cb105e9cac76ff4433065aa2b2f2abc2"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:52.411664 24673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "5d5afd3b8c7247be81ecafe4fd7562a0"
format_stamp: "Formatted at 2026-06-19 18:48:52 on dist-test-slave-fljv"
server_key: "73fa8e2d88135b2e8c0780a38b87f3fa"
server_key_iv: "cb105e9cac76ff4433065aa2b2f2abc2"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:52.417323 24673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 18:48:52.446731 24689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:52.448846 24673 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260619 18:48:52.449402 24673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "5d5afd3b8c7247be81ecafe4fd7562a0"
format_stamp: "Formatted at 2026-06-19 18:48:52 on dist-test-slave-fljv"
server_key: "73fa8e2d88135b2e8c0780a38b87f3fa"
server_key_iv: "cb105e9cac76ff4433065aa2b2f2abc2"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:52.449649 24673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:52.470368 24673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:52.474066 24673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:52.474349 24673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:52.475092 24673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:52.476209 24673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:52.476315 24673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:52.476415 24673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:52.476473 24673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:52.489732 24673 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:37271
I20260619 18:48:52.490396 24802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:37271 every 8 connection(s)
I20260619 18:48:52.491222 24673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:48:52.500774 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 24673
I20260619 18:48:52.500913 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:48:52.501255 13227 external_mini_cluster.cc:1428] Setting key 59d0a407a2397104a62daa89a1add9d0
Jun 19 18:48:52 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894932, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:48:52.512254 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.495831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:35233 (local address 127.12.234.254:37695)
0619 18:48:52.495969 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:52.495974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:52.496452 (+ 478us) server_negotiation.cc:407] Connection header received
0619 18:48:52.497776 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:52.497781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:52.497836 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:52.497887 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:52.499946 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.500767 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.503895 (+ 3128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.504265 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.504338 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.507450 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:52.507454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:52.507457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:52.507485 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:52.509116 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:52.509768 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:52.509771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:52.509773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:52.509861 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:52.510283 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:52.510286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:52.510288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:52.510507 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:52.510588 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:52.511950 (+ 1362us) server_negotiation.cc:299] Negotiation successful
0619 18:48:52.512087 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:48:52.513504 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.494617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:37695 (local address 127.12.234.194:35233)
0619 18:48:52.496265 (+ 1648us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:52.496331 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:52.497493 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:52.498552 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:52.498562 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:52.498946 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:52.499748 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.499770 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.500982 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.500986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.503744 (+ 2758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.503758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.504556 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.504559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.504778 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.505488 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:52.505509 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:52.507235 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:52.509252 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:52.509259 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:52.509280 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:52.509632 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:52.509963 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:52.509966 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:52.509968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:52.510158 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:52.510638 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:52.510645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:52.511060 (+ 415us) client_negotiation.cc:769] Sending connection context
0619 18:48:52.511275 (+ 215us) client_negotiation.cc:240] Negotiation successful
0619 18:48:52.511627 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1406,"thread_start_us":152,"threads_started":1}
I20260619 18:48:52.515035 24803 heartbeater.cc:347] Connected to a master server at 127.12.234.254:37695
I20260619 18:48:52.515290 24803 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:52.515853 24803 heartbeater.cc:511] Master 127.12.234.254:37695 requested a full tablet report, sending...
I20260619 18:48:52.516927 24453 ts_manager.cc:194] Registered new tserver with Master: 5d5afd3b8c7247be81ecafe4fd7562a0 (127.12.234.194:37271)
I20260619 18:48:52.517740 24453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:35233
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:48:52.547060 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:55251/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37695
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:36703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:48:52.691768 24808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:48:52.692250 24808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:48:52.692381 24808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:48:52.697840 24808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:48:52.697916 24808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:48:52.698010 24808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:48:52.704372 24808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:36703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.234.212:55251/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:37695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:48:52.705873 24808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:48:52.707377 24808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Jun 19 18:48:52 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894932, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
W20260619 18:48:52.731653 24816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:52.731752 24814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:48:52.731671 24813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:48:52.733250 24808 server_base.cc:1034] running on GCE node
I20260619 18:48:52.733844 24808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:48:52.734772 24808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:48:52.736059 24808 hybrid_clock.cc:648] HybridClock initialized: now 1781894932735991 us; error 87 us; skew 500 ppm
Jun 19 18:48:52 dist-test-slave-fljv krb5kdc[23977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894932, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:48:52.739356 24808 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:48:52.740255 24808 webserver.cc:460] Webserver started at http://127.12.234.195:44411/ using document root <none> and password file <none>
I20260619 18:48:52.740829 24808 fs_manager.cc:359] Metadata directory not provided
I20260619 18:48:52.740897 24808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:48:52.741115 24808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 18:48:52 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894932, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:48:52.765637 24808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "e871bb94f5204703b03c5ab33e819982"
format_stamp: "Formatted at 2026-06-19 18:48:52 on dist-test-slave-fljv"
server_key: "f933d9b757fe2cfec408858e494c001e"
server_key_iv: "25475fac4e4e0ac1832fd4c383fda3ee"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:52.766435 24808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "e871bb94f5204703b03c5ab33e819982"
format_stamp: "Formatted at 2026-06-19 18:48:52 on dist-test-slave-fljv"
server_key: "f933d9b757fe2cfec408858e494c001e"
server_key_iv: "25475fac4e4e0ac1832fd4c383fda3ee"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:52.778363 24808 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.002s sys 0.003s
I20260619 18:48:52.802652 24823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:52.804531 24808 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.000s sys 0.004s
I20260619 18:48:52.804689 24808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "e871bb94f5204703b03c5ab33e819982"
format_stamp: "Formatted at 2026-06-19 18:48:52 on dist-test-slave-fljv"
server_key: "f933d9b757fe2cfec408858e494c001e"
server_key_iv: "25475fac4e4e0ac1832fd4c383fda3ee"
server_key_version: "kuduclusterkey@0"
I20260619 18:48:52.804841 24808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:48:52.824311 24808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:48:52.833029 24808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:48:52.833292 24808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:48:52.834158 24808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:48:52.835312 24808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:48:52.835421 24808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:52.835521 24808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:48:52.835580 24808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:48:52.850159 24808 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:42497
I20260619 18:48:52.851438 24808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:48:52.852064 24936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:42497 every 8 connection(s)
I20260619 18:48:52.856801 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 24808
I20260619 18:48:52.856920 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:48:52.857237 13227 external_mini_cluster.cc:1428] Setting key d319f39d7dd406d4ee22afa463662a34
Jun 19 18:48:52 dist-test-slave-fljv krb5kdc[23977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894932, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:48:52.885178 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.854761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:46057 (local address 127.12.234.254:37695)
0619 18:48:52.854884 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:52.854888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:52.862802 (+ 7914us) server_negotiation.cc:407] Connection header received
0619 18:48:52.864167 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:52.864171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:52.864231 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:52.864283 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:52.866083 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.866942 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.869132 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.869504 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.869582 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.881019 (+ 11437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:52.881024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:52.881027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:52.881060 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:52.882653 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:52.883312 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:52.883315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:52.883317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:52.883401 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:52.883790 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:52.883793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:52.883795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:52.884035 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:52.884104 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:52.884883 (+ 779us) server_negotiation.cc:299] Negotiation successful
0619 18:48:52.885022 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:48:52.886972 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.855245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:37695 (local address 127.12.234.195:46057)
0619 18:48:52.862601 (+ 7356us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:52.862687 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:52.863890 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:52.864763 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:52.864774 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:52.865159 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:48:52.865895 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.865918 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.867558 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.867561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.868989 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.869002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.871508 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.871513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.871731 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.879054 (+ 7323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:48:52.879077 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:48:52.880749 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:48:52.882796 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:52.882803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:52.882822 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:52.883198 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:52.883500 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:48:52.883502 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:48:52.883504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:48:52.883689 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:48:52.884181 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:48:52.884187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:48:52.884563 (+ 376us) client_negotiation.cc:769] Sending connection context
0619 18:48:52.885355 (+ 792us) client_negotiation.cc:240] Negotiation successful
0619 18:48:52.885739 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7044,"thread_start_us":141,"threads_started":1}
I20260619 18:48:52.888430 24937 heartbeater.cc:347] Connected to a master server at 127.12.234.254:37695
I20260619 18:48:52.888700 24937 heartbeater.cc:464] Registering TS with master...
I20260619 18:48:52.889307 24937 heartbeater.cc:511] Master 127.12.234.254:37695 requested a full tablet report, sending...
I20260619 18:48:52.890501 24453 ts_manager.cc:194] Registered new tserver with Master: e871bb94f5204703b03c5ab33e819982 (127.12.234.195:42497)
I20260619 18:48:52.891048 24453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:46057
I20260619 18:48:52.894848 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:48:52.907766 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.897348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47432 (local address 127.12.234.254:37695)
0619 18:48:52.897516 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:52.897521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:52.898137 (+ 616us) server_negotiation.cc:407] Connection header received
0619 18:48:52.898331 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:52.898335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:52.898384 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:52.898431 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:48:52.899392 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.900216 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.901461 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.901805 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.901904 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.903885 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:48:52.903890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:48:52.903893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:48:52.903926 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:48:52.905682 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:52.906249 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:52.906253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:52.906255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:52.906347 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:48:52.906686 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:48:52.906690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:48:52.906692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:48:52.906926 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:48:52.907044 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:52.907435 (+ 391us) server_negotiation.cc:299] Negotiation successful
0619 18:48:52.907585 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 18:48:52.912205 24453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:48:52.916879 24453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:48:52.937897 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.928677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33984 (local address 127.12.234.195:42497)
0619 18:48:52.929772 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:52.929785 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:52.931188 (+ 1403us) server_negotiation.cc:407] Connection header received
0619 18:48:52.931394 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:52.931399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:52.932362 (+ 963us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:52.932494 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:52.932892 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.934019 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.936075 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.936929 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.937029 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.937171 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:52.937637 (+ 466us) server_negotiation.cc:299] Negotiation successful
0619 18:48:52.937744 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":989,"thread_start_us":105,"threads_started":1}
I20260619 18:48:52.938381 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.928292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:42497 (local address 127.0.0.1:33984)
0619 18:48:52.931095 (+ 2803us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:52.931108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:52.931264 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:52.932653 (+ 1389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:52.932656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:52.932673 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:52.932774 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.932779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.934151 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.934154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.935910 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.935930 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.937263 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.937266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.937423 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.937465 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 18:48:52.938063 (+ 598us) client_negotiation.cc:240] Negotiation successful
0619 18:48:52.938227 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2743,"mutex_wait_us":95,"spinlock_wait_cycles":4224,"thread_start_us":82,"threads_started":1}
I20260619 18:48:52.942142 24870 tablet_service.cc:1467] Processing CreateTablet for tablet 6904f837f11940658bea79c5cbad4c2f (DEFAULT_TABLE table=test-table [id=f374f57ed34a4c3f95e60ce8cd942746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:48:52.942833 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.927229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:37271 (local address 127.0.0.1:47018)
0619 18:48:52.929911 (+ 2682us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:52.929924 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:52.930013 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:52.931217 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:52.931220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:52.931238 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:52.931331 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.931337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.934210 (+ 2873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.934213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.936411 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.936421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.942313 (+ 5892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.942317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.942433 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.942464 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 18:48:52.942536 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 18:48:52.942620 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2623,"mutex_wait_us":800,"thread_start_us":93,"threads_started":1}
I20260619 18:48:52.943476 24870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6904f837f11940658bea79c5cbad4c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:52.943861 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.927389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47018 (local address 127.12.234.194:37271)
0619 18:48:52.930287 (+ 2898us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:52.930299 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:52.930326 (+ 27us) server_negotiation.cc:407] Connection header received
0619 18:48:52.930404 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:52.930410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:52.930643 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:52.931410 (+ 767us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:52.931576 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.932779 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.937836 (+ 5057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.941062 (+ 3226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.943338 (+ 2276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.943518 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:52.943617 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 18:48:52.943723 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2764,"thread_start_us":124,"threads_started":1}
I20260619 18:48:52.945465 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.927113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:46795 (local address 127.0.0.1:51114)
0619 18:48:52.929282 (+ 2169us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:52.929338 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:52.929470 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:52.930851 (+ 1381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:52.930857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:52.930879 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:52.931026 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.931034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.934839 (+ 3805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.934842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.937677 (+ 2835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.937688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.944151 (+ 6463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.944155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.945166 (+ 1011us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.945192 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 18:48:52.945246 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 18:48:52.945327 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1987,"mutex_wait_us":1688,"thread_start_us":127,"threads_started":1}
I20260619 18:48:52.946502 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.927978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51114 (local address 127.12.234.193:46795)
0619 18:48:52.930409 (+ 2431us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:52.930415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:52.930439 (+ 24us) server_negotiation.cc:407] Connection header received
0619 18:48:52.930515 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:52.930520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:52.930713 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:52.930858 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:52.932921 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.934054 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.941219 (+ 7165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.942120 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.945988 (+ 3868us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.946154 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:52.946235 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 18:48:52.946345 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2322,"thread_start_us":78,"threads_started":1}
I20260619 18:48:52.949373 24599 tablet_service.cc:1467] Processing CreateTablet for tablet 6904f837f11940658bea79c5cbad4c2f (DEFAULT_TABLE table=test-table [id=f374f57ed34a4c3f95e60ce8cd942746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:48:52.949455 24733 tablet_service.cc:1467] Processing CreateTablet for tablet 6904f837f11940658bea79c5cbad4c2f (DEFAULT_TABLE table=test-table [id=f374f57ed34a4c3f95e60ce8cd942746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:48:52.950429 24599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6904f837f11940658bea79c5cbad4c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:52.950445 24733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6904f837f11940658bea79c5cbad4c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:48:52.951527 24955 tablet_bootstrap.cc:492] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: Bootstrap starting.
I20260619 18:48:52.955142 24955 tablet_bootstrap.cc:654] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:52.956110 24955 log.cc:824] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:52.956732 24956 tablet_bootstrap.cc:492] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: Bootstrap starting.
I20260619 18:48:52.958664 24955 tablet_bootstrap.cc:492] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: No bootstrap required, opened a new log
I20260619 18:48:52.958861 24955 ts_tablet_manager.cc:1397] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 18:48:52.959344 24956 tablet_bootstrap.cc:654] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:52.960160 24956 log.cc:824] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:52.962718 24958 tablet_bootstrap.cc:492] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5: Bootstrap starting.
I20260619 18:48:52.963263 24955 raft_consensus.cc:348] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } }
I20260619 18:48:52.963822 24955 raft_consensus.cc:374] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:52.963910 24955 raft_consensus.cc:729] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e871bb94f5204703b03c5ab33e819982, State: Initialized, Role: FOLLOWER
I20260619 18:48:52.964460 24955 consensus_queue.cc:260] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } }
I20260619 18:48:52.965235 24958 tablet_bootstrap.cc:654] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5: Neither blocks nor log segments found. Creating new log.
I20260619 18:48:52.965641 24955 ts_tablet_manager.cc:1428] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 18:48:52.966038 24937 heartbeater.cc:503] Master 127.12.234.254:37695 was elected leader, sending a full tablet report...
I20260619 18:48:52.966071 24958 log.cc:824] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5: Log is configured to *not* fsync() on all Append() calls
I20260619 18:48:52.968382 24956 tablet_bootstrap.cc:492] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: No bootstrap required, opened a new log
I20260619 18:48:52.968605 24956 ts_tablet_manager.cc:1397] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260619 18:48:52.972715 24956 raft_consensus.cc:348] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } }
I20260619 18:48:52.973135 24956 raft_consensus.cc:374] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:52.973237 24956 raft_consensus.cc:729] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5afd3b8c7247be81ecafe4fd7562a0, State: Initialized, Role: FOLLOWER
I20260619 18:48:52.973838 24956 consensus_queue.cc:260] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } }
I20260619 18:48:52.974969 24958 tablet_bootstrap.cc:492] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5: No bootstrap required, opened a new log
I20260619 18:48:52.975325 24956 ts_tablet_manager.cc:1428] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260619 18:48:52.975414 24958 ts_tablet_manager.cc:1397] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260619 18:48:52.975736 24803 heartbeater.cc:503] Master 127.12.234.254:37695 was elected leader, sending a full tablet report...
I20260619 18:48:52.979646 24963 raft_consensus.cc:482] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:48:52.979874 24963 raft_consensus.cc:504] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } }
I20260619 18:48:52.982577 24958 raft_consensus.cc:348] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } }
I20260619 18:48:52.982945 24963 leader_election.cc:288] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a14acc65f6249a2835450849a9d3fb5 (127.12.234.193:46795), e871bb94f5204703b03c5ab33e819982 (127.12.234.195:42497)
I20260619 18:48:52.986450 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.981331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:46795 (local address 127.12.234.194:58033)
0619 18:48:52.981577 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:52.981592 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:52.981672 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:52.982699 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:52.982702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:52.982721 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:52.982833 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.982840 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.983922 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.983926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.985099 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.985110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.986046 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.986049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.986140 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.986162 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:48:52.986251 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 18:48:52.986328 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260619 18:48:52.987069 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.981809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:58033 (local address 127.12.234.193:46795)
0619 18:48:52.981936 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:52.981941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:52.981960 (+ 19us) server_negotiation.cc:407] Connection header received
0619 18:48:52.982012 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:52.982015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:52.982064 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:52.982145 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:52.982962 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.983742 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.985257 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.985832 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.985940 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.985992 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:52.986883 (+ 891us) server_negotiation.cc:299] Negotiation successful
0619 18:48:52.986963 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 18:48:52.987421 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.982536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:51601 (local address 127.12.234.195:42497)
0619 18:48:52.982646 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:52.982650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:52.983352 (+ 702us) server_negotiation.cc:407] Connection header received
0619 18:48:52.983530 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:52.983534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:52.983578 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:52.983621 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:48:52.984000 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.984691 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.986200 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.986760 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.986841 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.986884 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:52.987237 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 18:48:52.987311 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:48:52.987677 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6904f837f11940658bea79c5cbad4c2f" candidate_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a14acc65f6249a2835450849a9d3fb5" is_pre_election: true
I20260619 18:48:52.987800 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:52.982165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:42497 (local address 127.12.234.194:51601)
0619 18:48:52.983269 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:52.983282 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:48:52.983418 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:48:52.983742 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:48:52.983744 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:48:52.983760 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:48:52.983867 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.983872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.984837 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.984840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.986049 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:48:52.986058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:48:52.986998 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:48:52.987001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:48:52.987097 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:52.987117 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 18:48:52.987582 (+ 465us) client_negotiation.cc:240] Negotiation successful
0619 18:48:52.987657 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"thread_start_us":118,"threads_started":1}
I20260619 18:48:52.988353 24958 raft_consensus.cc:374] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:48:52.988333 24875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6904f837f11940658bea79c5cbad4c2f" candidate_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e871bb94f5204703b03c5ab33e819982" is_pre_election: true
I20260619 18:48:52.988494 24958 raft_consensus.cc:729] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a14acc65f6249a2835450849a9d3fb5, State: Initialized, Role: FOLLOWER
I20260619 18:48:52.988622 24875 raft_consensus.cc:2454] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d5afd3b8c7247be81ecafe4fd7562a0 in term 0.
I20260619 18:48:52.989096 24692 leader_election.cc:302] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d5afd3b8c7247be81ecafe4fd7562a0, e871bb94f5204703b03c5ab33e819982; no voters:
I20260619 18:48:52.989269 24958 consensus_queue.cc:260] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } }
I20260619 18:48:52.989416 24962 raft_consensus.cc:2781] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:48:52.989509 24962 raft_consensus.cc:482] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:48:52.989562 24962 raft_consensus.cc:3037] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:52.989723 24620 raft_consensus.cc:2454] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d5afd3b8c7247be81ecafe4fd7562a0 in term 0.
I20260619 18:48:52.990824 24962 raft_consensus.cc:504] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } }
I20260619 18:48:52.991431 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6904f837f11940658bea79c5cbad4c2f" candidate_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a14acc65f6249a2835450849a9d3fb5"
I20260619 18:48:52.991539 24620 raft_consensus.cc:3037] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:52.992049 24958 ts_tablet_manager.cc:1428] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260619 18:48:52.992439 24667 heartbeater.cc:503] Master 127.12.234.254:37695 was elected leader, sending a full tablet report...
I20260619 18:48:52.992730 24620 raft_consensus.cc:2454] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d5afd3b8c7247be81ecafe4fd7562a0 in term 1.
W20260619 18:48:52.992779 24804 tablet.cc:2369] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:48:52.993067 24693 leader_election.cc:302] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a14acc65f6249a2835450849a9d3fb5, 5d5afd3b8c7247be81ecafe4fd7562a0; no voters:
I20260619 18:48:52.993223 24963 raft_consensus.cc:2781] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:48:52.993407 24963 raft_consensus.cc:686] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 1 LEADER]: Becoming Leader. State: Replica: 5d5afd3b8c7247be81ecafe4fd7562a0, State: Running, Role: LEADER
I20260619 18:48:52.993713 24963 consensus_queue.cc:237] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } }
I20260619 18:48:52.994522 24962 leader_election.cc:288] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [CANDIDATE]: Term 1 election: Requested vote from peers 2a14acc65f6249a2835450849a9d3fb5 (127.12.234.193:46795), e871bb94f5204703b03c5ab33e819982 (127.12.234.195:42497)
I20260619 18:48:52.994612 24875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6904f837f11940658bea79c5cbad4c2f" candidate_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e871bb94f5204703b03c5ab33e819982"
I20260619 18:48:52.994756 24875 raft_consensus.cc:3037] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:48:52.996086 24875 raft_consensus.cc:2454] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d5afd3b8c7247be81ecafe4fd7562a0 in term 1.
I20260619 18:48:52.998167 24452 catalog_manager.cc:5495] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d5afd3b8c7247be81ecafe4fd7562a0 (127.12.234.194). New cstate: current_term: 1 leader_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d5afd3b8c7247be81ecafe4fd7562a0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 } health_report { overall_health: UNKNOWN } } }
I20260619 18:48:53.013506 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:53.009578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.12.234.194:37271)
0619 18:48:53.009701 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:48:53.009707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:48:53.009764 (+ 57us) server_negotiation.cc:407] Connection header received
0619 18:48:53.009943 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:48:53.009946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:48:53.010002 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:48:53.010090 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:48:53.010348 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:53.010958 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:53.011952 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:48:53.012227 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:48:53.012314 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:48:53.012631 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:48:53.013109 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:48:53.013219 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 18:48:53.013296 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 18:48:53.013356 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:48:53.020144 24620 raft_consensus.cc:1261] T 6904f837f11940658bea79c5cbad4c2f P 2a14acc65f6249a2835450849a9d3fb5 [term 1 FOLLOWER]: Refusing update from remote peer 5d5afd3b8c7247be81ecafe4fd7562a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:48:53.020144 24875 raft_consensus.cc:1261] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [term 1 FOLLOWER]: Refusing update from remote peer 5d5afd3b8c7247be81ecafe4fd7562a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:48:53.021129 24966 consensus_queue.cc:1035] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e871bb94f5204703b03c5ab33e819982" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 42497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:48:53.021387 24963 consensus_queue.cc:1035] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a14acc65f6249a2835450849a9d3fb5" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 46795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:48:53.028237 24969 mvcc.cc:205] Tried to move back new op lower bound from 7298641645643153408 to 7298641645547970560. Current Snapshot: MvccSnapshot[applied={T|T < 7298641645643153408}]
I20260619 18:48:53.032722 24971 mvcc.cc:205] Tried to move back new op lower bound from 7298641645643153408 to 7298641645547970560. Current Snapshot: MvccSnapshot[applied={T|T < 7298641645643153408}]
I20260619 18:48:53.041319 24970 mvcc.cc:205] Tried to move back new op lower bound from 7298641645643153408 to 7298641645547970560. Current Snapshot: MvccSnapshot[applied={T|T < 7298641645643153408}]
I20260619 18:48:53.044633 24452 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:48:53.044836 24452 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:48:53.047648 24452 catalog_manager.cc:5782] T 00000000000000000000000000000000 P deb95da8a0024736b4f9079e9110a40f: Sending DeleteTablet for 3 replicas of tablet 6904f837f11940658bea79c5cbad4c2f
I20260619 18:48:53.048471 24599 tablet_service.cc:1514] Processing DeleteTablet for tablet 6904f837f11940658bea79c5cbad4c2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:48:53 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:51114
I20260619 18:48:53.048857 24733 tablet_service.cc:1514] Processing DeleteTablet for tablet 6904f837f11940658bea79c5cbad4c2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:48:53 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47018
I20260619 18:48:53.049736 24870 tablet_service.cc:1514] Processing DeleteTablet for tablet 6904f837f11940658bea79c5cbad4c2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:48:53 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:33984
I20260619 18:48:53.051152 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 24534
I20260619 18:48:53.055753 24982 tablet_replica.cc:330] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: stopping tablet replica
I20260619 18:48:53.056165 24982 raft_consensus.cc:2229] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:48:53.056547 24981 tablet_replica.cc:330] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: stopping tablet replica
I20260619 18:48:53.056869 24981 raft_consensus.cc:2229] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:48:53.057366 24981 raft_consensus.cc:2258] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:48:53.057904 24982 raft_consensus.cc:2258] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:48:53.059625 24982 ts_tablet_manager.cc:1905] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:48:53.062623 24982 ts_tablet_manager.cc:1918] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:48:53.062729 24982 log.cc:1196] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/6904f837f11940658bea79c5cbad4c2f
I20260619 18:48:53.063069 24982 ts_tablet_manager.cc:1939] T 6904f837f11940658bea79c5cbad4c2f P e871bb94f5204703b03c5ab33e819982: Deleting consensus metadata
I20260619 18:48:53.064455 24439 catalog_manager.cc:4841] TS e871bb94f5204703b03c5ab33e819982 (127.12.234.195:42497): tablet 6904f837f11940658bea79c5cbad4c2f (table test-table [id=f374f57ed34a4c3f95e60ce8cd942746]) successfully deleted
I20260619 18:48:53.065733 24981 ts_tablet_manager.cc:1905] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:48:53.068431 24981 ts_tablet_manager.cc:1918] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:48:53.068540 24981 log.cc:1196] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/6904f837f11940658bea79c5cbad4c2f
I20260619 18:48:53.068847 24981 ts_tablet_manager.cc:1939] T 6904f837f11940658bea79c5cbad4c2f P 5d5afd3b8c7247be81ecafe4fd7562a0: Deleting consensus metadata
I20260619 18:48:53.070201 24439 catalog_manager.cc:4841] TS 5d5afd3b8c7247be81ecafe4fd7562a0 (127.12.234.194:37271): tablet 6904f837f11940658bea79c5cbad4c2f (table test-table [id=f374f57ed34a4c3f95e60ce8cd942746]) successfully deleted
W20260619 18:48:53.075559 24440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:46795 (error 108)
I20260619 18:48:53.076459 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 24673
I20260619 18:48:53.080539 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:48:53.080174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:46795 (local address 127.0.0.1:51122)
0619 18:48:53.080375 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 18:48:53.080449 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:46795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260619 18:48:53.080729 24440 catalog_manager.cc:4584] TS 2a14acc65f6249a2835450849a9d3fb5 (127.12.234.193:46795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6904f837f11940658bea79c5cbad4c2f: Network error: Client connection negotiation failed: client connection to 127.12.234.193:46795: connect: Connection refused (error 111)
I20260619 18:48:53.083069 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 24808
I20260619 18:48:53.089427 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 24421
I20260619 18:48:53.096556 13227 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 18:48:53.431547 13227 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 18:48:53.431672 13227 mini_ranger.cc:66] Stopping Ranger...
I20260619 18:48:53.766826 13227 mini_ranger.cc:68] Stopped Ranger
2026-06-19 18:48:53.767 UTC [24007] LOG: received smart shutdown request
2026-06-19 18:48:53.770 UTC [24007] LOG: background worker "logical replication launcher" (PID 24016) exited with exit code 1
2026-06-19 18:48:53.770 UTC [24011] LOG: shutting down
2026-06-19 18:48:53.924 UTC [24007] LOG: database system is shut down
2026-06-19T18:48:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46735 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:48:54 dist-test-slave-fljv krb5kdc[24992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:48:54 dist-test-slave-fljv krb5kdc[24992](info): set up 2 sockets
Jun 19 18:48:54 dist-test-slave-fljv krb5kdc[24992](info): commencing operation
krb5kdc: starting...
W20260619 18:48:54.982024 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:48:55 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:48:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:48:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.234.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.234.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/rangeradmin_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/rangeradmin_127.12.234.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.234.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.234.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/rangerlookup_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/rangerlookup_127.12.234.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.234.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.234.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/HTTP_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/HTTP_127.12.234.212@KRBTEST.COM.keytab.
I20260619 18:48:55.140753 13227 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJa0qyg/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/postgres -l logfile start
2026-06-19 18:48:57.592 UTC [25022] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 18:48:57.592 UTC [25022] LOG: listening on IPv4 address "127.12.234.212", port 52687
2026-06-19 18:48:57.595 UTC [25022] LOG: listening on Unix socket "/tmp/.s.PGSQL.52687"
2026-06-19 18:48:57.615 UTC [25025] LOG: database system was shut down at 2026-06-19 18:48:56 UTC
2026-06-19 18:48:57.618 UTC [25022] LOG: database system is ready to accept connections
I20260619 18:48:58.485790 13227 mini_postgres.cc:96] Postgres bound to 52687
2026-06-19 18:48:58.491 UTC [25033] FATAL: database "slave" does not exist
127.12.234.212:52687 - accepting connections
I20260619 18:48:58.492093 13227 mini_ranger.cc:161] Starting Ranger...
I20260619 18:48:58.506210 13227 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 18:48:58.982125 13227 mini_ranger.cc:87] Created ranger Postgres database
I20260619 18:48:58.982228 13227 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-admin
2026-06-19 18:48:59,240 [I] DB FLAVOR :POSTGRES
2026-06-19 18:48:59,241 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 18:48:59,241 [I] Checking connection..
2026-06-19 18:48:59,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 18:48:59,476 [I] Checking connection passed.
2026-06-19 18:48:59,476 [I] --------- Verifying version history table ---------
2026-06-19 18:48:59,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 18:48:59,732 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 18:48:59,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 18:48:59,999 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 18:48:59,999 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 18:48:59,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 18:49:00.259 UTC [25097] WARNING: there is no transaction in progress
2026-06-19 18:49:00,265 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 18:49:00,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 18:49:00,513 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 18:49:00,513 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 18:49:00,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 18:49:00,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 18:49:01,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fljv.c.gcp-upstream.internal','N') ;"
2026-06-19 18:49:01,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 18:49:01,484 [I] Table x_portal_user does not exist in database ranger
2026-06-19 18:49:01,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 18:49:01,744 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 18:49:01,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 18:49:01,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 18:49:02,241 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 18:49:02,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 18:49:03.229 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.246 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.264 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.281 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.298 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.320 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.386 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.393 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.408 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.418 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.430 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.444 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.454 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.461 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.467 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:03.473 UTC [25205] WARNING: there is no transaction in progress
2026-06-19 18:49:04,135 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 18:49:04,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 18:49:04,366 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 18:49:04,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 18:49:04,607 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 18:49:04,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 18:49:04,858 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 18:49:04,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 18:49:05,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 18:49:05,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fljv.c.gcp-upstream.internal';"
2026-06-19 18:49:05,590 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 18:49:05,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 18:49:05,840 [I] DB_PATCHES have already been applied
I20260619 18:49:05.846017 13227 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJa0qyg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 18:49:05.846088 13227 mini_ranger.cc:193] Using host: 127.12.234.212
I20260619 18:49:05.849493 13227 mini_ranger.cc:227] Ranger admin URL: http://127.12.234.212:59973
Jun 19, 2026 6:49:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:49:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:49:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 6:49:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 6:49:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 6:49:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 6:49:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 18:49:06.848663 13227 mini_ranger.cc:160] Time spent starting Ranger: real 8.357s user 0.008s sys 0.001s
Jun 19, 2026 6:49:26 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 6:49:26 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 19, 2026 6:49:26 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 6:49:26 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 6:49:27 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 18:49:27.935658 13227 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.234.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.234.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/rangerkms_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.234.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/rangerkms_127.12.234.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.234.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.234.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.234.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/HTTP_127.12.234.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.234.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/HTTP_127.12.234.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 18:49:28 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894968, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 18:49:28.032688 13227 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 18:49:28.043066 13227 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 18:49:28.894927 13227 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 18:49:28.895008 13227 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-kms
I20260619 18:49:28.895030 13227 mini_ranger_kms.cc:227] Using postgres at 127.12.234.212:52687
2026-06-19 18:49:29,048 [I] DB FLAVOR :POSTGRES
2026-06-19 18:49:29,048 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 18:49:29,048 [I] Checking connection
2026-06-19 18:49:29,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 18:49:29,296 [I] connection success
2026-06-19 18:49:29,296 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 18:49:29,296 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 18:49:29,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 18:49:29,536 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 18:49:29,536 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 18:49:29,536 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 18:49:29,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.212:52687/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 18:49:29,787 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 18:49:30.301975 13227 mini_ranger_kms.cc:310] Created kms service
I20260619 18:49:30.519333 13227 mini_ranger_kms.cc:326] Created kudu user
I20260619 18:49:30.563961 13227 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 18:49:30 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894968, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:30.960578 13227 mini_ranger_kms.cc:384] Added ranger policy
I20260619 18:49:30.960754 13227 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJa0qyg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJa0qyg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 18:49:30.960783 13227 mini_ranger_kms.cc:240] Using host: 127.12.234.212
I20260619 18:49:30.965030 13227 mini_ranger_kms.cc:276] Ranger KMS PID: 25385
I20260619 18:49:30.965173 13227 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.234.212:51303
Jun 19, 2026 6:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:49:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 6:49:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 6:49:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 6:49:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 18:49:33.038358 13227 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.006s user 0.015s sys 0.123s
I20260619 18:49:33.038480 13227 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 18:49:33.038543 13227 mini_ranger_kms.cc:401] 127.12.234.212:51303/kms/v1/keys
Jun 19 18:49:33 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781894973, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 18:49:33 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894973, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.212@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
Jun 19 18:49:34 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894973, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.212@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
Jun 19 18:49:36 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:49:36.341354 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41463
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:41463
--ranger_config_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:51303/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:36.454810 25436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:36.455101 25436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:36.455140 25436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:36.459434 25436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:49:36.459506 25436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:36.459525 25436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:36.459540 25436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:49:36.459555 25436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:49:36.465400 25436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.234.212:51303/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:41463
--ranger_config_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:36.466584 25436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:36.467715 25436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:36.476593 25441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:36.476603 25442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:36.476797 25436 server_base.cc:1034] running on GCE node
W20260619 18:49:36.477208 25444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:36.477838 25436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:36.479101 25436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:36.480288 25436 hybrid_clock.cc:648] HybridClock initialized: now 1781894976480260 us; error 63 us; skew 500 ppm
Jun 19 18:49:36 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894976, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:36.483235 25436 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:49:36.484290 25436 webserver.cc:460] Webserver started at http://127.12.234.254:33199/ using document root <none> and password file <none>
I20260619 18:49:36.484836 25436 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:36.484894 25436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:36.485090 25436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 18:49:36 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894976, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:36.533326 25436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "02e0d40a3d04452997f5d25997e194f6"
format_stamp: "Formatted at 2026-06-19 18:49:36 on dist-test-slave-fljv"
server_key: "020b45eba25c31c7d7bd53c1d9b75e55"
server_key_iv: "2f36c45930e11836233da956e6efc9ab"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:36.533977 25436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "02e0d40a3d04452997f5d25997e194f6"
format_stamp: "Formatted at 2026-06-19 18:49:36 on dist-test-slave-fljv"
server_key: "020b45eba25c31c7d7bd53c1d9b75e55"
server_key_iv: "2f36c45930e11836233da956e6efc9ab"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:36.538051 25436 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260619 18:49:36.557360 25451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:36.558837 25436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:49:36.558970 25436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "02e0d40a3d04452997f5d25997e194f6"
format_stamp: "Formatted at 2026-06-19 18:49:36 on dist-test-slave-fljv"
server_key: "020b45eba25c31c7d7bd53c1d9b75e55"
server_key_iv: "2f36c45930e11836233da956e6efc9ab"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:36.559115 25436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:36.613368 25436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:36.616351 25436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:36.616528 25436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:36.624230 25436 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:41463
I20260619 18:49:36.624243 25503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:41463 every 8 connection(s)
I20260619 18:49:36.625267 25436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:49:36.628696 25504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:36.630647 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 25436
I20260619 18:49:36.630759 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:49:36.631038 13227 external_mini_cluster.cc:1428] Setting key 28216fc188761bedfd9779ebf39d747f
I20260619 18:49:36.636015 25504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Bootstrap starting.
I20260619 18:49:36.638832 25504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Neither blocks nor log segments found. Creating new log.
Jun 19 18:49:36 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:36.639708 25504 log.cc:824] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:36.642113 25504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: No bootstrap required, opened a new log
I20260619 18:49:36.646056 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:36.632493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48672 (local address 127.12.234.254:41463)
0619 18:49:36.632911 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:36.632926 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:36.632964 (+ 38us) server_negotiation.cc:407] Connection header received
0619 18:49:36.633603 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:36.633619 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:36.633975 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:36.634327 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:36.635101 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:36.636084 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:36.636998 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:36.637473 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:36.637620 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:36.640090 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:36.640098 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:36.640116 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:36.640149 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:36.642732 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:36.643279 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:36.643285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:36.643291 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:36.643436 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:36.643716 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:36.643720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:36.643721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:36.644252 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:36.644457 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:36.644744 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 18:49:36.645045 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":123,"threads_started":1}
I20260619 18:49:36.646119 25504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } }
I20260619 18:49:36.646448 25504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:36.646517 25504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e0d40a3d04452997f5d25997e194f6, State: Initialized, Role: FOLLOWER
I20260619 18:49:36.647025 25504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } }
I20260619 18:49:36.647188 25504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:49:36.647279 25504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:49:36.647382 25504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:36.648626 25504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } }
I20260619 18:49:36.649025 25504 leader_election.cc:302] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02e0d40a3d04452997f5d25997e194f6; no voters:
I20260619 18:49:36.649354 25504 leader_election.cc:288] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:49:36.649441 25509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:49:36.649616 25509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 1 LEADER]: Becoming Leader. State: Replica: 02e0d40a3d04452997f5d25997e194f6, State: Running, Role: LEADER
I20260619 18:49:36.649993 25509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } }
I20260619 18:49:36.650710 25504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:49:36.651234 25509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02e0d40a3d04452997f5d25997e194f6. Latest consensus state: current_term: 1 leader_uuid: "02e0d40a3d04452997f5d25997e194f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } } }
I20260619 18:49:36.651443 25509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:36.652307 25510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02e0d40a3d04452997f5d25997e194f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } } }
I20260619 18:49:36.652410 25510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:36.652700 25516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:49:36.652889 25504 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 18:49:36.658984 25516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:49:36.675259 25516 catalog_manager.cc:1347] Generated new cluster ID: c141653e53644fb899e4a45cecd55151
I20260619 18:49:36.675340 25516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:49:36.701836 25516 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:49:36.703471 25516 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:49:36.717662 25516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Generated new TSK 0
I20260619 18:49:36.718845 25516 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:49:37.397378 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:51303/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41463
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:37.543794 25549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:37.544211 25549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:37.544301 25549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:37.550671 25549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:37.550829 25549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:37.550980 25549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:49:37.559409 25549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.234.212:51303/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:37.561174 25549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:37.562811 25549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:37.575570 25559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:37.575570 25557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:37.575871 25556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:37.576485 25549 server_base.cc:1034] running on GCE node
I20260619 18:49:37.576959 25549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:37.577766 25549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:37.578961 25549 hybrid_clock.cc:648] HybridClock initialized: now 1781894977578932 us; error 47 us; skew 500 ppm
Jun 19 18:49:37 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894977, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:37.582073 25549 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:49:37.583091 25549 webserver.cc:460] Webserver started at http://127.12.234.193:34713/ using document root <none> and password file <none>
I20260619 18:49:37.583662 25549 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:37.583724 25549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:37.583932 25549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 18:49:37 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894977, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:37.601691 25549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "4dd17da4e5d84ec097565f541cf51621"
format_stamp: "Formatted at 2026-06-19 18:49:37 on dist-test-slave-fljv"
server_key: "e66a90af529dad04b5ba2f98fe2a9229"
server_key_iv: "20af941441b75fbaf9242f732236a1ee"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:37.602722 25549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "4dd17da4e5d84ec097565f541cf51621"
format_stamp: "Formatted at 2026-06-19 18:49:37 on dist-test-slave-fljv"
server_key: "e66a90af529dad04b5ba2f98fe2a9229"
server_key_iv: "20af941441b75fbaf9242f732236a1ee"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:37.620012 25549 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.004s sys 0.000s
I20260619 18:49:37.649271 25568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:37.651155 25549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:49:37.651288 25549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "4dd17da4e5d84ec097565f541cf51621"
format_stamp: "Formatted at 2026-06-19 18:49:37 on dist-test-slave-fljv"
server_key: "e66a90af529dad04b5ba2f98fe2a9229"
server_key_iv: "20af941441b75fbaf9242f732236a1ee"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:37.651425 25549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:37.685987 25549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:37.690024 25549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:37.690331 25549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:37.691123 25549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:37.692267 25549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:37.692374 25549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:37.692495 25549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:37.692587 25549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:37.706331 25549 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:35307
I20260619 18:49:37.707636 25549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:49:37.709219 25681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:35307 every 8 connection(s)
I20260619 18:49:37.714417 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 25549
I20260619 18:49:37.714546 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:49:37.714867 13227 external_mini_cluster.cc:1428] Setting key cc40ba8578b7872e9f9005b2d400b803
Jun 19 18:49:37 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894977, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:37.733258 25685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:37.713028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:60075 (local address 127.12.234.254:41463)
0619 18:49:37.718772 (+ 5744us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:37.718782 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:37.718804 (+ 22us) server_negotiation.cc:407] Connection header received
0619 18:49:37.720344 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:37.720348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:37.720404 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:37.720486 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:37.722121 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:37.723060 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:37.724827 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:37.725184 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:37.725263 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:37.728490 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:37.728494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:37.728497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:37.728564 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:37.730725 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:37.731406 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:37.731410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:37.731412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:37.731505 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:37.731901 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:37.731904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:37.731906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:37.732133 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:37.732199 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:37.732927 (+ 728us) server_negotiation.cc:299] Negotiation successful
0619 18:49:37.733062 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2539,"thread_start_us":106,"threads_started":1}
I20260619 18:49:37.735347 25684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:37.711762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41463 (local address 127.12.234.193:60075)
0619 18:49:37.713883 (+ 2121us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:37.713946 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:37.719913 (+ 5967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:37.720834 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:37.720846 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:37.721211 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:37.721949 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:37.721969 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:37.723206 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:37.723209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:37.724689 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:37.724701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:37.725510 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:37.725514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:37.725727 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:37.726479 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:37.726501 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:37.728269 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:37.730884 (+ 2615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:37.730890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:37.730911 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:37.731290 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:37.731607 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:37.731610 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:37.731612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:37.731803 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:37.732291 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:37.732299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:37.732647 (+ 348us) client_negotiation.cc:769] Sending connection context
0619 18:49:37.733771 (+ 1124us) client_negotiation.cc:240] Negotiation successful
0619 18:49:37.734145 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1843,"thread_start_us":161,"threads_started":1}
I20260619 18:49:37.737133 25682 heartbeater.cc:347] Connected to a master server at 127.12.234.254:41463
I20260619 18:49:37.737402 25682 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:37.737993 25682 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
I20260619 18:49:37.739910 25468 ts_manager.cc:194] Registered new tserver with Master: 4dd17da4e5d84ec097565f541cf51621 (127.12.234.193:35307)
I20260619 18:49:37.741439 25468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:60075
Jun 19 18:49:37 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781894977, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:49:37.786024 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:51303/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41463
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:37.943409 25688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:37.944339 25688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:37.944497 25688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:37.951118 25688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:37.951340 25688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:37.951486 25688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:49:37.959879 25688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.234.212:51303/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:37.961658 25688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:37.963289 25688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:37.976939 25695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:37.978020 25697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:37.978344 25688 server_base.cc:1034] running on GCE node
W20260619 18:49:37.980508 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:37.981016 25688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:37.981709 25688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:37.982867 25688 hybrid_clock.cc:648] HybridClock initialized: now 1781894977982815 us; error 71 us; skew 500 ppm
Jun 19 18:49:37 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894977, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:37.985549 25688 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:49:37.986518 25688 webserver.cc:460] Webserver started at http://127.12.234.194:44523/ using document root <none> and password file <none>
I20260619 18:49:37.987123 25688 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:37.987212 25688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:37.987439 25688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 18:49:37 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894977, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:38.004014 25688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "b8aabea8c5ba463c8e08b2b4a76c4124"
format_stamp: "Formatted at 2026-06-19 18:49:37 on dist-test-slave-fljv"
server_key: "09e06d78d6fe9df5a0127152578cbdf3"
server_key_iv: "d1ead00dc176a9d9a4b0afd2f5f1e888"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:38.004655 25688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "b8aabea8c5ba463c8e08b2b4a76c4124"
format_stamp: "Formatted at 2026-06-19 18:49:37 on dist-test-slave-fljv"
server_key: "09e06d78d6fe9df5a0127152578cbdf3"
server_key_iv: "d1ead00dc176a9d9a4b0afd2f5f1e888"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:38.011499 25688 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260619 18:49:38.031051 25704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:38.032882 25688 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260619 18:49:38.033017 25688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "b8aabea8c5ba463c8e08b2b4a76c4124"
format_stamp: "Formatted at 2026-06-19 18:49:37 on dist-test-slave-fljv"
server_key: "09e06d78d6fe9df5a0127152578cbdf3"
server_key_iv: "d1ead00dc176a9d9a4b0afd2f5f1e888"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:38.033160 25688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:38.068953 25688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:38.072887 25688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:38.073107 25688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:38.073946 25688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:38.075163 25688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:38.075232 25688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:38.075291 25688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:38.075321 25688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:38.086268 25688 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:38773
I20260619 18:49:38.087639 25688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:49:38.091957 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 25688
I20260619 18:49:38.092087 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:49:38.092399 13227 external_mini_cluster.cc:1428] Setting key 23ca4752fcd4b7df8a385b787da697d9
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:49:38.108464 25817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:38773 every 8 connection(s)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
Jun 19 18:49:38 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894977, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:38.137305 25685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:38.109260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:55473 (local address 127.12.234.254:41463)
0619 18:49:38.109405 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:38.109410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:38.114788 (+ 5378us) server_negotiation.cc:407] Connection header received
0619 18:49:38.116136 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:38.116140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:38.116199 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:38.116263 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:38.118034 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:38.118849 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:38.120575 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:38.120899 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:38.120981 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:38.127159 (+ 6178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:38.127163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:38.127166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:38.127198 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:38.129030 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:38.129724 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:38.129728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:38.129730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:38.129823 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:38.130690 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:38.130694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:38.130696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:38.130941 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:38.131011 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:38.136907 (+ 5896us) server_negotiation.cc:299] Negotiation successful
0619 18:49:38.137078 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 18:49:38.139806 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:38.110224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41463 (local address 127.12.234.194:55473)
0619 18:49:38.114578 (+ 4354us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:38.114652 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:38.115848 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:38.116727 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:38.116738 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:38.117109 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:38.117850 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:38.117873 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:38.119007 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:38.119010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:38.120443 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:38.120454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:38.121192 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:38.121197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:38.121440 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:38.124884 (+ 3444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:38.124908 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:38.126910 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:38.129197 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:38.129203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:38.129236 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:38.129603 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:38.129935 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:38.129938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:38.129939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:38.130153 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:38.131091 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:38.131098 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:38.131477 (+ 379us) client_negotiation.cc:769] Sending connection context
0619 18:49:38.137889 (+ 6412us) client_negotiation.cc:240] Negotiation successful
0619 18:49:38.138430 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4067,"thread_start_us":167,"threads_started":1}
I20260619 18:49:38.141557 25818 heartbeater.cc:347] Connected to a master server at 127.12.234.254:41463
I20260619 18:49:38.141810 25818 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:38.142549 25818 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
I20260619 18:49:38.143790 25468 ts_manager.cc:194] Registered new tserver with Master: b8aabea8c5ba463c8e08b2b4a76c4124 (127.12.234.194:38773)
I20260619 18:49:38.144574 25468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:55473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:49:38.162500 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:51303/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41463
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 18:49:38 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894977, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
W20260619 18:49:38.347043 25823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:38.347499 25823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:38.347570 25823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:38.353601 25823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:38.353688 25823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:38.353775 25823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:49:38.361949 25823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.234.212:51303/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:41463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:38.363531 25823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:38.364894 25823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:38.377146 25831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:38.377170 25828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:38.377653 25829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:38.378139 25823 server_base.cc:1034] running on GCE node
I20260619 18:49:38.378747 25823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:38.379529 25823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:38.380712 25823 hybrid_clock.cc:648] HybridClock initialized: now 1781894978380681 us; error 46 us; skew 500 ppm
Jun 19 18:49:38 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894978, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:38.384433 25823 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:49:38.385850 25823 webserver.cc:460] Webserver started at http://127.12.234.195:40753/ using document root <none> and password file <none>
I20260619 18:49:38.386770 25823 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:38.386878 25823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:38.387197 25823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 18:49:38 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894978, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:38.404372 25823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "77abdc4f544e485681eb0b7e3ca7bb56"
format_stamp: "Formatted at 2026-06-19 18:49:38 on dist-test-slave-fljv"
server_key: "866c1d53304d7dfc41711c343e7ccd0f"
server_key_iv: "bc37b6f09392ec9ca89dd33eea7965da"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:38.405270 25823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "77abdc4f544e485681eb0b7e3ca7bb56"
format_stamp: "Formatted at 2026-06-19 18:49:38 on dist-test-slave-fljv"
server_key: "866c1d53304d7dfc41711c343e7ccd0f"
server_key_iv: "bc37b6f09392ec9ca89dd33eea7965da"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:38.411584 25823 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260619 18:49:38.438656 25838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:38.445230 25823 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.002s sys 0.000s
I20260619 18:49:38.445427 25823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "77abdc4f544e485681eb0b7e3ca7bb56"
format_stamp: "Formatted at 2026-06-19 18:49:38 on dist-test-slave-fljv"
server_key: "866c1d53304d7dfc41711c343e7ccd0f"
server_key_iv: "bc37b6f09392ec9ca89dd33eea7965da"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:38.445600 25823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:38.467962 25823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:38.471892 25823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:38.472121 25823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:38.472877 25823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:38.473951 25823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:38.474013 25823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:38.474066 25823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:38.474094 25823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:38.484781 25823 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:46175
I20260619 18:49:38.486048 25823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:49:38.486837 25951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:46175 every 8 connection(s)
I20260619 18:49:38.493643 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 25823
I20260619 18:49:38.493777 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:49:38.494100 13227 external_mini_cluster.cc:1428] Setting key ac4637791a6757d66b5b361e1456e725
Jun 19 18:49:38 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894978, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:38.508149 25685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:38.490380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:40685 (local address 127.12.234.254:41463)
0619 18:49:38.490496 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:38.490501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:38.494567 (+ 4066us) server_negotiation.cc:407] Connection header received
0619 18:49:38.496059 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:38.496062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:38.496120 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:38.496176 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:38.498033 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:38.498859 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:38.500621 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:38.500945 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:38.501043 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:38.503922 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:38.503927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:38.503931 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:38.503961 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:38.505574 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:38.506168 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:38.506172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:38.506174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:38.506291 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:38.506686 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:38.506689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:38.506691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:38.507097 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:38.507165 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:38.507853 (+ 688us) server_negotiation.cc:299] Negotiation successful
0619 18:49:38.507992 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:49:38.509857 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:38.489676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41463 (local address 127.12.234.195:40685)
0619 18:49:38.494299 (+ 4623us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:38.494383 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:38.495778 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:38.496650 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:38.496662 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:38.497084 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:38.497853 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:38.497874 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:38.499007 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:38.499010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:38.500491 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:38.500503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:38.501114 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:38.501119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:38.501328 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:38.501996 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:38.502018 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:38.503716 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:38.505698 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:38.505705 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:38.505724 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:38.506057 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:38.506381 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:38.506384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:38.506386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:38.506578 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:38.507251 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:38.507258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:38.507582 (+ 324us) client_negotiation.cc:769] Sending connection context
0619 18:49:38.508364 (+ 782us) client_negotiation.cc:240] Negotiation successful
0619 18:49:38.508714 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4281,"thread_start_us":141,"threads_started":1}
I20260619 18:49:38.511149 25952 heartbeater.cc:347] Connected to a master server at 127.12.234.254:41463
I20260619 18:49:38.511421 25952 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:38.512123 25952 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
I20260619 18:49:38.513363 25468 ts_manager.cc:194] Registered new tserver with Master: 77abdc4f544e485681eb0b7e3ca7bb56 (127.12.234.195:46175)
I20260619 18:49:38.514119 25468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:40685
I20260619 18:49:38.520047 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:49:38.520848 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 25549
I20260619 18:49:38.527801 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 25688
I20260619 18:49:38.533763 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 25823
I20260619 18:49:38.543839 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 25436
I20260619 18:49:38.549925 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41463
--webserver_interface=127.12.234.254
--webserver_port=33199
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:41463
--ranger_config_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:51303/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.234.212:51303/kms with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:38.660673 25955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:38.660953 25955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:38.660995 25955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:38.665189 25955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:49:38.665259 25955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:38.665277 25955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:38.665293 25955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:49:38.665306 25955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:49:38.671070 25955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.234.212:51303/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:41463
--ranger_config_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=33199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:38.672129 25955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:38.673128 25955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:38.681725 25960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:38.681747 25961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:38.681840 25955 server_base.cc:1034] running on GCE node
W20260619 18:49:38.681725 25963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:38.682509 25955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:38.683545 25955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:38.684707 25955 hybrid_clock.cc:648] HybridClock initialized: now 1781894978684683 us; error 38 us; skew 500 ppm
Jun 19 18:49:38 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894978, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:38.687317 25955 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:49:38.688246 25955 webserver.cc:460] Webserver started at http://127.12.234.254:33199/ using document root <none> and password file <none>
I20260619 18:49:38.688766 25955 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:38.688830 25955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 18:49:38 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894978, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:38.740067 25955 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:49:38.743011 25971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:38.744629 25955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:49:38.744783 25955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "02e0d40a3d04452997f5d25997e194f6"
format_stamp: "Formatted at 2026-06-19 18:49:36 on dist-test-slave-fljv"
server_key: "020b45eba25c31c7d7bd53c1d9b75e55"
server_key_iv: "2f36c45930e11836233da956e6efc9ab"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:38.745517 25955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:38.769213 25955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:38.772233 25955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:38.772403 25955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:38.779392 25955 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:41463
I20260619 18:49:38.779443 26023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:41463 every 8 connection(s)
I20260619 18:49:38.780516 25955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:49:38.784986 26024 sys_catalog.cc:263] Verifying existing consensus state
I20260619 18:49:38.786785 26024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Bootstrap starting.
I20260619 18:49:38.787070 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 25955
I20260619 18:49:38.787460 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:51303/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:35307
--local_ip_for_outbound_sockets=127.12.234.193
--tserver_master_addrs=127.12.234.254:41463
--webserver_port=34713
--webserver_interface=127.12.234.193
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.234.212:51303/kms with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:49:38.803516 26024 log.cc:824] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:38.810526 26024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:49:38.810976 26024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Bootstrap complete.
I20260619 18:49:38.815019 26024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } }
I20260619 18:49:38.815421 26024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e0d40a3d04452997f5d25997e194f6, State: Initialized, Role: FOLLOWER
I20260619 18:49:38.815924 26024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } }
I20260619 18:49:38.816059 26024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:49:38.816128 26024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:49:38.816221 26024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 1 FOLLOWER]: Advancing to term 2
I20260619 18:49:38.818082 26024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } }
I20260619 18:49:38.818424 26024 leader_election.cc:302] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02e0d40a3d04452997f5d25997e194f6; no voters:
I20260619 18:49:38.818709 26024 leader_election.cc:288] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 18:49:38.818848 26028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 18:49:38.819137 26028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [term 2 LEADER]: Becoming Leader. State: Replica: 02e0d40a3d04452997f5d25997e194f6, State: Running, Role: LEADER
I20260619 18:49:38.819451 26028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } }
I20260619 18:49:38.819898 26024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:49:38.820989 26030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02e0d40a3d04452997f5d25997e194f6. Latest consensus state: current_term: 2 leader_uuid: "02e0d40a3d04452997f5d25997e194f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } } }
I20260619 18:49:38.821030 26029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "02e0d40a3d04452997f5d25997e194f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e0d40a3d04452997f5d25997e194f6" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41463 } } }
I20260619 18:49:38.821115 26030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:38.821135 26029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:38.821518 26037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:49:38.824838 26037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:49:38.825377 26024 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 18:49:38.825464 26037 catalog_manager.cc:1259] Loaded cluster ID: c141653e53644fb899e4a45cecd55151
I20260619 18:49:38.825529 26037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:49:38.827831 26037 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:49:38.828481 26037 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Loaded TSK: 0
I20260619 18:49:38.829661 26037 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 18:49:38.906011 26026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:38.906433 26026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:38.906507 26026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:38.911739 26026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:38.911829 26026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:38.911918 26026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:49:38.917920 26026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.234.212:51303/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:35307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=34713
--tserver_master_addrs=127.12.234.254:41463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:38.919102 26026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:38.920356 26026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:38.930205 26064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:38.930375 26062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:38.930761 26061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:38.930711 26026 server_base.cc:1034] running on GCE node
I20260619 18:49:38.931237 26026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:38.931927 26026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:38.933105 26026 hybrid_clock.cc:648] HybridClock initialized: now 1781894978933074 us; error 39 us; skew 500 ppm
Jun 19 18:49:38 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894978, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:38.938943 26026 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:49:38.940074 26026 webserver.cc:460] Webserver started at http://127.12.234.193:34713/ using document root <none> and password file <none>
I20260619 18:49:38.940654 26026 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:38.940737 26026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 18:49:38 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894978, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:38.959833 26026 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 18:49:38.962605 26072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:38.964154 26026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:49:38.964368 26026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "4dd17da4e5d84ec097565f541cf51621"
format_stamp: "Formatted at 2026-06-19 18:49:37 on dist-test-slave-fljv"
server_key: "e66a90af529dad04b5ba2f98fe2a9229"
server_key_iv: "20af941441b75fbaf9242f732236a1ee"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:38.965047 26026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:39.003976 26026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:39.007367 26026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:39.007570 26026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:39.008227 26026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:39.009255 26026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:39.009307 26026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:39.009361 26026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:39.009383 26026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:39.019546 26026 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:35307
I20260619 18:49:39.019573 26185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:35307 every 8 connection(s)
I20260619 18:49:39.020804 26026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:49:39.025929 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 26026
I20260619 18:49:39.026437 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:51303/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:38773
--local_ip_for_outbound_sockets=127.12.234.194
--tserver_master_addrs=127.12.234.254:41463
--webserver_port=44523
--webserver_interface=127.12.234.194
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.234.212:51303/kms with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 18:49:39 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894978, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:39.044375 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:39.023329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41463 (local address 127.12.234.193:56011)
0619 18:49:39.024494 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:39.024552 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:39.025394 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:39.028480 (+ 3086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:39.028491 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:39.028883 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:39.029513 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:39.029530 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.031486 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.031490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:39.033013 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:39.033027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.034505 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.034511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:39.034721 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:39.035476 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:39.035499 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:39.037261 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:39.040065 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:39.040072 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:39.040091 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:39.040491 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:39.040949 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:39.040952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:39.040954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:39.041166 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:39.042068 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:39.042076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:39.042522 (+ 446us) client_negotiation.cc:769] Sending connection context
0619 18:49:39.042762 (+ 240us) client_negotiation.cc:240] Negotiation successful
0619 18:49:39.043137 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"thread_start_us":130,"threads_started":1}
I20260619 18:49:39.046507 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:39.023148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:56011 (local address 127.12.234.254:41463)
0619 18:49:39.023766 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:39.023789 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:39.024689 (+ 900us) server_negotiation.cc:407] Connection header received
0619 18:49:39.026845 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:39.026877 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:39.027366 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:39.027781 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:39.029708 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.031263 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.033186 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.033992 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.034210 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:39.037560 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:39.037567 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:39.037590 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:39.037629 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:39.039896 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:39.040645 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:39.040652 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:39.040658 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:39.040828 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:39.041300 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:39.041304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:39.041306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:39.041870 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:39.041998 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:39.045107 (+ 3109us) server_negotiation.cc:299] Negotiation successful
0619 18:49:39.045461 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":107,"threads_started":1}
I20260619 18:49:39.049463 26186 heartbeater.cc:347] Connected to a master server at 127.12.234.254:41463
I20260619 18:49:39.049734 26186 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:39.050420 26186 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
W20260619 18:49:39.053336 26186 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:41463 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 18:49:39.053488 26186 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:39.053714 26186 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
I20260619 18:49:39.054773 26186 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:39.054973 26186 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
W20260619 18:49:39.055799 26186 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 18:49:39.172960 26190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:39.173331 26190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:39.173411 26190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:39.177803 26190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:39.177878 26190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:39.177950 26190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:49:39.183749 26190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.234.212:51303/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:38773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=44523
--tserver_master_addrs=127.12.234.254:41463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:39.184940 26190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:39.185974 26190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:39.196424 26198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:39.196435 26195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:39.196434 26196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:39.197118 26190 server_base.cc:1034] running on GCE node
I20260619 18:49:39.197695 26190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:39.198673 26190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:39.199918 26190 hybrid_clock.cc:648] HybridClock initialized: now 1781894979199879 us; error 54 us; skew 500 ppm
Jun 19 18:49:39 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894979, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:39.203868 26190 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:49:39.205160 26190 webserver.cc:460] Webserver started at http://127.12.234.194:44523/ using document root <none> and password file <none>
I20260619 18:49:39.205952 26190 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:39.206069 26190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 18:49:39 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894979, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:39.231281 26190 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:49:39.238955 26206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:39.240520 26190 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260619 18:49:39.240664 26190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "b8aabea8c5ba463c8e08b2b4a76c4124"
format_stamp: "Formatted at 2026-06-19 18:49:37 on dist-test-slave-fljv"
server_key: "09e06d78d6fe9df5a0127152578cbdf3"
server_key_iv: "d1ead00dc176a9d9a4b0afd2f5f1e888"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:39.241334 26190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:39.257522 26190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:39.261317 26190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:39.261554 26190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:39.262401 26190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:39.263540 26190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:39.263615 26190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:39.263726 26190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:39.263758 26190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:39.275967 26190 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:38773
I20260619 18:49:39.276804 26319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:38773 every 8 connection(s)
I20260619 18:49:39.277338 26190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:49:39.281705 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 26190
I20260619 18:49:39.282171 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.212:51303/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:46175
--local_ip_for_outbound_sockets=127.12.234.195
--tserver_master_addrs=127.12.234.254:41463
--webserver_port=40753
--webserver_interface=127.12.234.195
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.234.212:51303/kms with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 18:49:39 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894979, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:39.300935 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:39.282048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:53029 (local address 127.12.234.254:41463)
0619 18:49:39.282235 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:39.282241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:39.282606 (+ 365us) server_negotiation.cc:407] Connection header received
0619 18:49:39.283867 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:39.283871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:39.283937 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:39.283991 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:39.285667 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.286529 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.288307 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.288669 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.288750 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:39.296402 (+ 7652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:39.296408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:39.296411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:39.296446 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:39.298136 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:39.298866 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:39.298870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:39.298872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:39.299021 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:39.299488 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:39.299492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:39.299493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:39.299734 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:39.299809 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:39.300592 (+ 783us) server_negotiation.cc:299] Negotiation successful
0619 18:49:39.300743 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:49:39.302909 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:39.280842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41463 (local address 127.12.234.194:53029)
0619 18:49:39.282435 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:39.282493 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:39.283597 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:39.284428 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:39.284437 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:39.284793 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:39.285484 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:39.285504 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.286709 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.286713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:39.288163 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:39.288175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.288970 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.288974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:39.289177 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:39.294366 (+ 5189us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:39.294393 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:39.296142 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:39.298313 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:39.298320 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:39.298342 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:39.298734 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:39.299148 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:39.299151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:39.299153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:39.299367 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:39.299895 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:39.299936 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:39.300297 (+ 361us) client_negotiation.cc:769] Sending connection context
0619 18:49:39.301207 (+ 910us) client_negotiation.cc:240] Negotiation successful
0619 18:49:39.301594 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1354,"thread_start_us":142,"threads_started":1}
I20260619 18:49:39.304410 26320 heartbeater.cc:347] Connected to a master server at 127.12.234.254:41463
I20260619 18:49:39.304692 26320 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:39.305306 26320 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
W20260619 18:49:39.307086 26320 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:41463 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 18:49:39.307221 26320 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:39.307422 26320 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
I20260619 18:49:39.308300 26320 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:39.308483 26320 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
W20260619 18:49:39.309283 26320 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 18:49:39.440173 26323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:39.440446 26323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:39.440507 26323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:39.445142 26323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:39.445214 26323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:39.445297 26323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:49:39.452750 26323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.234.212:51303/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:46175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=40753
--tserver_master_addrs=127.12.234.254:41463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:39.454170 26323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:39.455662 26323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:49:39.466266 26323 server_base.cc:1034] running on GCE node
W20260619 18:49:39.466217 26332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:39.466143 26329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:39.466146 26330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:39.466894 26323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:39.467669 26323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:39.468918 26323 hybrid_clock.cc:648] HybridClock initialized: now 1781894979468876 us; error 46 us; skew 500 ppm
Jun 19 18:49:39 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894979, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:39.471779 26323 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:49:39.472729 26323 webserver.cc:460] Webserver started at http://127.12.234.195:40753/ using document root <none> and password file <none>
I20260619 18:49:39.473297 26323 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:39.473379 26323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 18:49:39 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894979, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:39.499842 26323 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:49:39.502501 26340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:39.504122 26323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:49:39.504338 26323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "77abdc4f544e485681eb0b7e3ca7bb56"
format_stamp: "Formatted at 2026-06-19 18:49:38 on dist-test-slave-fljv"
server_key: "866c1d53304d7dfc41711c343e7ccd0f"
server_key_iv: "bc37b6f09392ec9ca89dd33eea7965da"
server_key_version: "kuduclusterkey@0"
I20260619 18:49:39.505151 26323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:39.519397 26323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:39.523054 26323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:39.523260 26323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:39.524048 26323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:39.525038 26323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:39.525090 26323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:39.525144 26323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:39.525170 26323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:39.535377 26323 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:46175
I20260619 18:49:39.535648 26453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:46175 every 8 connection(s)
I20260619 18:49:39.536762 26323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:49:39.540606 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 26323
Jun 19 18:49:39 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894979, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:39.554697 26458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:39.542529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36022 (local address 127.12.234.254:41463)
0619 18:49:39.543616 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:39.543626 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:39.544054 (+ 428us) server_negotiation.cc:407] Connection header received
0619 18:49:39.544098 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:39.544101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:39.544147 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:39.544227 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:39.545310 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.546426 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.547657 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.547920 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.548014 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:39.550052 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:39.550056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:39.550059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:39.550091 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:39.551915 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:39.552514 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:39.552519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:39.552521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:39.552609 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:39.552926 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:39.552929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:39.552931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:39.553352 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:39.553559 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:39.554377 (+ 818us) server_negotiation.cc:299] Negotiation successful
0619 18:49:39.554508 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1013,"thread_start_us":89,"threads_started":1}
I20260619 18:49:39.555418 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:39.541900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:46181 (local address 127.12.234.254:41463)
0619 18:49:39.542039 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:39.542044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:39.542062 (+ 18us) server_negotiation.cc:407] Connection header received
0619 18:49:39.543424 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:39.543429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:39.543488 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:39.543575 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:39.545255 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.546095 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.547401 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.547628 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.547725 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:39.550580 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:39.550584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:39.550587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:39.550636 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:39.552927 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:39.553582 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:39.553585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:39.553587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:39.553666 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:39.554105 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:39.554109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:39.554111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:39.554343 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:39.554418 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:39.555167 (+ 749us) server_negotiation.cc:299] Negotiation successful
0619 18:49:39.555280 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":63}
I20260619 18:49:39.556919 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:39.540701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:41463 (local address 127.12.234.195:46181)
0619 18:49:39.541365 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:39.541436 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:39.543137 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:39.543915 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:39.543927 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:39.544317 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:39.545100 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:39.545116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.546250 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.546254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:39.547278 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:39.547289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:39.547793 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:39.547797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:39.548046 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:39.548737 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:39.548760 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:39.550316 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:39.553051 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:39.553058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:39.553081 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:39.553477 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:39.553773 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:39.553776 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:39.553777 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:39.553996 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:39.554487 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:39.554494 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:39.554881 (+ 387us) client_negotiation.cc:769] Sending connection context
0619 18:49:39.555609 (+ 728us) client_negotiation.cc:240] Negotiation successful
0619 18:49:39.555992 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":150,"threads_started":1}
I20260619 18:49:39.559491 26454 heartbeater.cc:347] Connected to a master server at 127.12.234.254:41463
I20260619 18:49:39.559826 26454 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:39.560899 26454 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
W20260619 18:49:39.562652 26454 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:41463 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 18:49:39.562793 26454 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:39.563009 26454 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
I20260619 18:49:39.564802 26454 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:39.565002 26454 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
W20260619 18:49:39.565864 26454 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 18:49:39 dist-test-slave-fljv krb5kdc[24992](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781894979, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:40.056057 26186 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:40.056349 26186 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
I20260619 18:49:40.057425 25988 ts_manager.cc:194] Registered new tserver with Master: 4dd17da4e5d84ec097565f541cf51621 (127.12.234.193:35307)
I20260619 18:49:40.058773 25988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:56011
I20260619 18:49:40.309544 26320 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:40.309876 26320 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
I20260619 18:49:40.310930 25988 ts_manager.cc:194] Registered new tserver with Master: b8aabea8c5ba463c8e08b2b4a76c4124 (127.12.234.194:38773)
I20260619 18:49:40.311771 25988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:53029
Jun 19 18:49:40 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894979, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
I20260619 18:49:40.566118 26454 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:40.566452 26454 heartbeater.cc:511] Master 127.12.234.254:41463 requested a full tablet report, sending...
I20260619 18:49:40.567268 25988 ts_manager.cc:194] Registered new tserver with Master: 77abdc4f544e485681eb0b7e3ca7bb56 (127.12.234.195:46175)
I20260619 18:49:40.567874 25988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:46181
I20260619 18:49:40.571995 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:49:40.580977 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.573489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36038 (local address 127.12.234.254:41463)
0619 18:49:40.573722 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:40.573727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:40.573837 (+ 110us) server_negotiation.cc:407] Connection header received
0619 18:49:40.573963 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:40.573966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:40.574015 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:40.574087 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:40.574850 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.575407 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.576246 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.576498 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.576601 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.577907 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:40.577911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:40.577914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:40.577942 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:40.579372 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:40.579777 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:40.579781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:40.579783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:40.579874 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:40.580123 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:40.580126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:40.580128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:40.580312 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:40.580432 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:40.580661 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 18:49:40.580785 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260619 18:49:40.584687 25988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:49:40.588300 25988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:49:40.607599 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.600224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.12.234.193:35307)
0619 18:49:40.600609 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:40.600618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:40.602695 (+ 2077us) server_negotiation.cc:407] Connection header received
0619 18:49:40.602896 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:40.602901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:40.603098 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:40.603238 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:40.603695 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.604658 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.605781 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.606553 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.606687 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.606856 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:40.607295 (+ 439us) server_negotiation.cc:299] Negotiation successful
0619 18:49:40.607415 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":82,"threads_started":1}
I20260619 18:49:40.608404 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.598770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:35307 (local address 127.0.0.1:52048)
0619 18:49:40.602540 (+ 3770us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:40.602587 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:40.602731 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:40.603243 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:40.603249 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:40.603273 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:40.603427 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.603435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.604773 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.604776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.605658 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.605668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.606934 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.606938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.607051 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.607098 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 18:49:40.608109 (+ 1011us) client_negotiation.cc:240] Negotiation successful
0619 18:49:40.608264 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3603,"thread_start_us":141,"threads_started":1}
I20260619 18:49:40.612375 26104 tablet_service.cc:1467] Processing CreateTablet for tablet eb77ae391a2048f9ac5c5787ea1dfaa6 (DEFAULT_TABLE table=test-table [id=582d86ca5126492fadb422b5e07a0987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:49:40.613783 26104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb77ae391a2048f9ac5c5787ea1dfaa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:40.620286 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.599736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.12.234.195:46175)
0619 18:49:40.608479 (+ 8743us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:40.608486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:40.608511 (+ 25us) server_negotiation.cc:407] Connection header received
0619 18:49:40.608588 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:40.608593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:40.608791 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:40.608945 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:40.614278 (+ 5333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.615428 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.617025 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.617908 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.618052 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.618242 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:40.620013 (+ 1771us) server_negotiation.cc:299] Negotiation successful
0619 18:49:40.620132 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8625,"thread_start_us":71,"threads_started":1}
I20260619 18:49:40.620554 26482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.599532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:46175 (local address 127.0.0.1:47848)
0619 18:49:40.606363 (+ 6831us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:40.606383 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:40.606482 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:40.609079 (+ 2597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:40.609083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:40.609101 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:40.609197 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.609203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.615595 (+ 6392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.615599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.616851 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.616867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.619396 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.619404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.619526 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.619556 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 18:49:40.619638 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 18:49:40.619737 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6744,"thread_start_us":107,"threads_started":1}
I20260619 18:49:40.623543 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.600207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.12.234.194:38773)
0619 18:49:40.600786 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:40.600792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:40.606767 (+ 5975us) server_negotiation.cc:407] Connection header received
0619 18:49:40.606994 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:40.607001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:40.607252 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:40.607406 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:40.607867 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.608974 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.621784 (+ 12810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.622686 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.622790 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.622914 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:40.623263 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 18:49:40.623389 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":103,"threads_started":1}
I20260619 18:49:40.624167 26484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.599866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:38773 (local address 127.0.0.1:33360)
0619 18:49:40.606661 (+ 6795us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:40.606674 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:40.606844 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:40.607606 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:40.607609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:40.607628 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:40.607720 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.607726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.609345 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.609348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.621603 (+ 12255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.621618 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.623017 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.623021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.623111 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.623133 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:49:40.623940 (+ 807us) client_negotiation.cc:240] Negotiation successful
0619 18:49:40.624034 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6738,"thread_start_us":66,"threads_started":1}
I20260619 18:49:40.624271 26371 tablet_service.cc:1467] Processing CreateTablet for tablet eb77ae391a2048f9ac5c5787ea1dfaa6 (DEFAULT_TABLE table=test-table [id=582d86ca5126492fadb422b5e07a0987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:49:40.625598 26371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb77ae391a2048f9ac5c5787ea1dfaa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:40.627558 26237 tablet_service.cc:1467] Processing CreateTablet for tablet eb77ae391a2048f9ac5c5787ea1dfaa6 (DEFAULT_TABLE table=test-table [id=582d86ca5126492fadb422b5e07a0987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:49:40.628720 26237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb77ae391a2048f9ac5c5787ea1dfaa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:40.642921 26487 tablet_bootstrap.cc:492] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: Bootstrap starting.
I20260619 18:49:40.644766 26489 tablet_bootstrap.cc:492] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621: Bootstrap starting.
I20260619 18:49:40.647089 26487 tablet_bootstrap.cc:654] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:40.649011 26489 tablet_bootstrap.cc:654] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:40.650094 26487 log.cc:824] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:40.653003 26487 tablet_bootstrap.cc:492] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: No bootstrap required, opened a new log
I20260619 18:49:40.653235 26487 ts_tablet_manager.cc:1397] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.004s
I20260619 18:49:40.655771 26488 tablet_bootstrap.cc:492] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: Bootstrap starting.
I20260619 18:49:40.657523 26487 raft_consensus.cc:348] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } }
I20260619 18:49:40.657956 26487 raft_consensus.cc:374] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:40.658043 26487 raft_consensus.cc:729] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8aabea8c5ba463c8e08b2b4a76c4124, State: Initialized, Role: FOLLOWER
I20260619 18:49:40.658519 26488 tablet_bootstrap.cc:654] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:40.658736 26487 consensus_queue.cc:260] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } }
I20260619 18:49:40.659500 26488 log.cc:824] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:40.660005 26487 ts_tablet_manager.cc:1428] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 18:49:40.660421 26320 heartbeater.cc:503] Master 127.12.234.254:41463 was elected leader, sending a full tablet report...
I20260619 18:49:40.663477 26489 log.cc:824] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:40.672606 26488 tablet_bootstrap.cc:492] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: No bootstrap required, opened a new log
I20260619 18:49:40.672880 26488 ts_tablet_manager.cc:1397] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260619 18:49:40.677047 26488 raft_consensus.cc:348] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } }
I20260619 18:49:40.677443 26488 raft_consensus.cc:374] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:40.677510 26488 raft_consensus.cc:729] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77abdc4f544e485681eb0b7e3ca7bb56, State: Initialized, Role: FOLLOWER
I20260619 18:49:40.678046 26488 consensus_queue.cc:260] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } }
I20260619 18:49:40.679293 26488 ts_tablet_manager.cc:1428] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260619 18:49:40.679710 26454 heartbeater.cc:503] Master 127.12.234.254:41463 was elected leader, sending a full tablet report...
I20260619 18:49:40.681506 26489 tablet_bootstrap.cc:492] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621: No bootstrap required, opened a new log
I20260619 18:49:40.681787 26489 ts_tablet_manager.cc:1397] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621: Time spent bootstrapping tablet: real 0.037s user 0.006s sys 0.000s
I20260619 18:49:40.686115 26489 raft_consensus.cc:348] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } }
I20260619 18:49:40.690783 26489 raft_consensus.cc:374] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:40.690945 26489 raft_consensus.cc:729] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd17da4e5d84ec097565f541cf51621, State: Initialized, Role: FOLLOWER
I20260619 18:49:40.691612 26489 consensus_queue.cc:260] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } }
I20260619 18:49:40.692868 26186 heartbeater.cc:503] Master 127.12.234.254:41463 was elected leader, sending a full tablet report...
I20260619 18:49:40.693192 26489 ts_tablet_manager.cc:1428] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621: Time spent starting tablet: real 0.011s user 0.002s sys 0.004s
Jun 19 18:49:40 dist-test-slave-fljv krb5kdc[24992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894979, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for HTTP/127.12.234.212@KRBTEST.COM
W20260619 18:49:40.777257 26187 tablet.cc:2369] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:49:40.779675 26321 tablet.cc:2369] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:49:40.789067 26455 tablet.cc:2369] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:49:40.936530 26491 raft_consensus.cc:482] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:49:40.936775 26491 raft_consensus.cc:504] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } }
I20260619 18:49:40.938289 26491 leader_election.cc:288] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dd17da4e5d84ec097565f541cf51621 (127.12.234.193:35307), 77abdc4f544e485681eb0b7e3ca7bb56 (127.12.234.195:46175)
I20260619 18:49:40.943842 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.938687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:49507 (local address 127.12.234.193:35307)
0619 18:49:40.938830 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:40.938836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:40.939097 (+ 261us) server_negotiation.cc:407] Connection header received
0619 18:49:40.939301 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:40.939305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:40.939354 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:40.939417 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:40.939867 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.940682 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.942464 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.943083 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.943186 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.943243 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:40.943610 (+ 367us) server_negotiation.cc:299] Negotiation successful
0619 18:49:40.943702 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 18:49:40.944499 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.938589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:35307 (local address 127.12.234.194:49507)
0619 18:49:40.938998 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:40.939013 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:40.939177 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:40.939568 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:40.939571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:40.939591 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:40.939733 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.939740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.940838 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.940842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.942248 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.942261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.943341 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.943345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.943462 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.943488 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 18:49:40.944247 (+ 759us) client_negotiation.cc:240] Negotiation successful
0619 18:49:40.944336 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":111,"threads_started":1}
I20260619 18:49:40.945334 26122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb77ae391a2048f9ac5c5787ea1dfaa6" candidate_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dd17da4e5d84ec097565f541cf51621" is_pre_election: true
I20260619 18:49:40.945698 26122 raft_consensus.cc:2454] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8aabea8c5ba463c8e08b2b4a76c4124 in term 0.
I20260619 18:49:40.946378 26209 leader_election.cc:302] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd17da4e5d84ec097565f541cf51621, b8aabea8c5ba463c8e08b2b4a76c4124; no voters:
I20260619 18:49:40.946709 26491 raft_consensus.cc:2781] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:49:40.946779 26491 raft_consensus.cc:482] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:49:40.946815 26491 raft_consensus.cc:3037] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:40.948149 26491 raft_consensus.cc:504] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } }
I20260619 18:49:40.948540 26491 leader_election.cc:288] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [CANDIDATE]: Term 1 election: Requested vote from peers 4dd17da4e5d84ec097565f541cf51621 (127.12.234.193:35307), 77abdc4f544e485681eb0b7e3ca7bb56 (127.12.234.195:46175)
I20260619 18:49:40.949209 26122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb77ae391a2048f9ac5c5787ea1dfaa6" candidate_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dd17da4e5d84ec097565f541cf51621"
I20260619 18:49:40.949340 26122 raft_consensus.cc:3037] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:40.949532 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.939405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:44111 (local address 127.12.234.195:46175)
0619 18:49:40.939521 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:40.939525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:40.945106 (+ 5581us) server_negotiation.cc:407] Connection header received
0619 18:49:40.945317 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:40.945320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:40.945374 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:40.945423 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:40.945837 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.946619 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.948179 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.948784 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.948887 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.948954 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:40.949305 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 18:49:40.949398 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 18:49:40.949906 26501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.939029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:46175 (local address 127.12.234.194:44111)
0619 18:49:40.945001 (+ 5972us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:40.945017 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:40.945192 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:40.945567 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:40.945570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:40.945590 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:40.945702 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.945709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.946764 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.946767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.948030 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:40.948041 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.949054 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.949057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:40.949166 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.949188 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:49:40.949681 (+ 493us) client_negotiation.cc:240] Negotiation successful
0619 18:49:40.949764 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5882,"thread_start_us":95,"threads_started":1}
I20260619 18:49:40.950752 26122 raft_consensus.cc:2454] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8aabea8c5ba463c8e08b2b4a76c4124 in term 1.
I20260619 18:49:40.950821 26408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb77ae391a2048f9ac5c5787ea1dfaa6" candidate_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" is_pre_election: true
I20260619 18:49:40.951156 26408 raft_consensus.cc:2454] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8aabea8c5ba463c8e08b2b4a76c4124 in term 0.
I20260619 18:49:40.951366 26209 leader_election.cc:302] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd17da4e5d84ec097565f541cf51621, b8aabea8c5ba463c8e08b2b4a76c4124; no voters:
I20260619 18:49:40.951609 26491 raft_consensus.cc:2781] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:49:40.951589 26407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb77ae391a2048f9ac5c5787ea1dfaa6" candidate_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77abdc4f544e485681eb0b7e3ca7bb56"
I20260619 18:49:40.951699 26407 raft_consensus.cc:3037] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:40.951824 26491 raft_consensus.cc:686] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 1 LEADER]: Becoming Leader. State: Replica: b8aabea8c5ba463c8e08b2b4a76c4124, State: Running, Role: LEADER
I20260619 18:49:40.952176 26491 consensus_queue.cc:237] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } }
I20260619 18:49:40.952966 26407 raft_consensus.cc:2454] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8aabea8c5ba463c8e08b2b4a76c4124 in term 1.
I20260619 18:49:40.955597 25988 catalog_manager.cc:5495] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8aabea8c5ba463c8e08b2b4a76c4124 (127.12.234.194). New cstate: current_term: 1 leader_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8aabea8c5ba463c8e08b2b4a76c4124" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 38773 } health_report { overall_health: HEALTHY } } }
I20260619 18:49:40.980485 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:40.976829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33376 (local address 127.12.234.194:38773)
0619 18:49:40.976947 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:40.976951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:40.976977 (+ 26us) server_negotiation.cc:407] Connection header received
0619 18:49:40.977166 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:40.977169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:40.977226 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:40.977326 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:49:40.977597 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.978104 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.979007 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:40.979223 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:40.979314 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:40.979623 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:49:40.980067 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:49:40.980180 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:40.980247 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 18:49:40.980310 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 18:49:40.986734 26407 raft_consensus.cc:1261] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [term 1 FOLLOWER]: Refusing update from remote peer b8aabea8c5ba463c8e08b2b4a76c4124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:49:40.986780 26122 raft_consensus.cc:1261] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 4dd17da4e5d84ec097565f541cf51621 [term 1 FOLLOWER]: Refusing update from remote peer b8aabea8c5ba463c8e08b2b4a76c4124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:49:40.987550 26491 consensus_queue.cc:1035] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77abdc4f544e485681eb0b7e3ca7bb56" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:49:40.987804 26503 consensus_queue.cc:1035] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd17da4e5d84ec097565f541cf51621" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:49:40.995756 26510 mvcc.cc:205] Tried to move back new op lower bound from 7298641842113998848 to 7298641841983049728. Current Snapshot: MvccSnapshot[applied={T|T < 7298641842113998848}]
I20260619 18:49:41.005090 26511 mvcc.cc:205] Tried to move back new op lower bound from 7298641842113998848 to 7298641841983049728. Current Snapshot: MvccSnapshot[applied={T|T < 7298641842113998848}]
I20260619 18:49:41.006096 25988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:49:41.006286 25988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:49:41.008981 25988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02e0d40a3d04452997f5d25997e194f6: Sending DeleteTablet for 3 replicas of tablet eb77ae391a2048f9ac5c5787ea1dfaa6
I20260619 18:49:41.009829 26104 tablet_service.cc:1514] Processing DeleteTablet for tablet eb77ae391a2048f9ac5c5787ea1dfaa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:41 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:52048
I20260619 18:49:41.010102 26371 tablet_service.cc:1514] Processing DeleteTablet for tablet eb77ae391a2048f9ac5c5787ea1dfaa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:41 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47848
I20260619 18:49:41.010387 26237 tablet_service.cc:1514] Processing DeleteTablet for tablet eb77ae391a2048f9ac5c5787ea1dfaa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:41 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:33360
I20260619 18:49:41.010818 26521 tablet_replica.cc:330] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: stopping tablet replica
I20260619 18:49:41.011236 26521 raft_consensus.cc:2229] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:49:41.011646 26521 raft_consensus.cc:2258] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:49:41.011761 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 26026
I20260619 18:49:41.014096 26521 ts_tablet_manager.cc:1905] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:49:41.016993 26521 ts_tablet_manager.cc:1918] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:49:41.017185 26521 log.cc:1196] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/eb77ae391a2048f9ac5c5787ea1dfaa6
I20260619 18:49:41.017573 26521 ts_tablet_manager.cc:1939] T eb77ae391a2048f9ac5c5787ea1dfaa6 P b8aabea8c5ba463c8e08b2b4a76c4124: Deleting consensus metadata
I20260619 18:49:41.018945 25974 catalog_manager.cc:4841] TS b8aabea8c5ba463c8e08b2b4a76c4124 (127.12.234.194:38773): tablet eb77ae391a2048f9ac5c5787ea1dfaa6 (table test-table [id=582d86ca5126492fadb422b5e07a0987]) successfully deleted
W20260619 18:49:41.020819 25974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:35307 (error 108)
I20260619 18:49:41.021070 26520 tablet_replica.cc:330] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: stopping tablet replica
I20260619 18:49:41.021442 26520 raft_consensus.cc:2229] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:49:41.021821 26520 raft_consensus.cc:2258] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:49:41.022235 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 26190
I20260619 18:49:41.023205 26484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:41.021943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:35307 (local address 127.0.0.1:52062)
0619 18:49:41.022151 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:41.023104 (+ 953us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:35307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260619 18:49:41.023486 25974 catalog_manager.cc:4584] TS 4dd17da4e5d84ec097565f541cf51621 (127.12.234.193:35307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb77ae391a2048f9ac5c5787ea1dfaa6: Network error: Client connection negotiation failed: client connection to 127.12.234.193:35307: connect: Connection refused (error 111)
I20260619 18:49:41.025147 26520 ts_tablet_manager.cc:1905] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:49:41.027591 26520 ts_tablet_manager.cc:1918] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:49:41.027693 26520 log.cc:1196] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/eb77ae391a2048f9ac5c5787ea1dfaa6
I20260619 18:49:41.027967 26520 ts_tablet_manager.cc:1939] T eb77ae391a2048f9ac5c5787ea1dfaa6 P 77abdc4f544e485681eb0b7e3ca7bb56: Deleting consensus metadata
I20260619 18:49:41.028882 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 26323
I20260619 18:49:41.034401 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 25955
I20260619 18:49:41.043023 13227 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 18:49:41.378018 13227 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 18:49:41.378149 13227 mini_ranger.cc:66] Stopping Ranger...
I20260619 18:49:41.713279 13227 mini_ranger.cc:68] Stopped Ranger
2026-06-19 18:49:41.713 UTC [25022] LOG: received smart shutdown request
2026-06-19 18:49:41.717 UTC [25022] LOG: background worker "logical replication launcher" (PID 25031) exited with exit code 1
2026-06-19 18:49:41.717 UTC [25026] LOG: shutting down
2026-06-19 18:49:41.876 UTC [25022] LOG: database system is shut down
2026-06-19T18:49:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47960 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:49:42 dist-test-slave-fljv krb5kdc[26531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:49:42 dist-test-slave-fljv krb5kdc[26531](info): set up 2 sockets
Jun 19 18:49:42 dist-test-slave-fljv krb5kdc[26531](info): commencing operation
krb5kdc: starting...
W20260619 18:49:42.950084 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:49:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:49:43.067080 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:34163
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:34163,127.12.234.253:40401,127.12.234.252:43699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:43.178460 26544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:43.178750 26544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:43.178790 26544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:43.182897 26544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:49:43.182966 26544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:43.182984 26544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:43.183001 26544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:49:43.183029 26544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:49:43.188548 26544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:34163,127.12.234.253:40401,127.12.234.252:43699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:34163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:43.189651 26544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:43.190716 26544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:43.199446 26550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:43.199484 26549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:43.199677 26544 server_base.cc:1034] running on GCE node
W20260619 18:49:43.199446 26552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:43.200354 26544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:43.201545 26544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:43.202786 26544 hybrid_clock.cc:648] HybridClock initialized: now 1781894983202759 us; error 41 us; skew 500 ppm
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:43.205740 26544 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:49:43.206736 26544 webserver.cc:460] Webserver started at http://127.12.234.254:46711/ using document root <none> and password file <none>
I20260619 18:49:43.207298 26544 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:43.207366 26544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:43.207585 26544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:49:43.209731 26544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "a5f84b5b64fe483aba053bc57ea8ab17"
format_stamp: "Formatted at 2026-06-19 18:49:43 on dist-test-slave-fljv"
server_key: "005302cc72356f6ebbaef7bb678fbb12"
server_key_iv: "a264bb7e183dcf2453d0eea58a39e043"
server_key_version: "encryptionkey@0"
I20260619 18:49:43.210354 26544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "a5f84b5b64fe483aba053bc57ea8ab17"
format_stamp: "Formatted at 2026-06-19 18:49:43 on dist-test-slave-fljv"
server_key: "005302cc72356f6ebbaef7bb678fbb12"
server_key_iv: "a264bb7e183dcf2453d0eea58a39e043"
server_key_version: "encryptionkey@0"
I20260619 18:49:43.214131 26544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:49:43.216799 26559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:43.218173 26544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:49:43.218360 26544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "a5f84b5b64fe483aba053bc57ea8ab17"
format_stamp: "Formatted at 2026-06-19 18:49:43 on dist-test-slave-fljv"
server_key: "005302cc72356f6ebbaef7bb678fbb12"
server_key_iv: "a264bb7e183dcf2453d0eea58a39e043"
server_key_version: "encryptionkey@0"
I20260619 18:49:43.218511 26544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:43.246470 26544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:43.249482 26544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:43.249675 26544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:43.256716 26544 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:34163
I20260619 18:49:43.256984 26611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:34163 every 8 connection(s)
I20260619 18:49:43.257726 26544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:49:43.260632 26612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:43.263620 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 26544
I20260619 18:49:43.263767 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:49:43.264026 13227 external_mini_cluster.cc:1428] Setting key 2a7928e6581f45449184dd914da59138
I20260619 18:49:43.264680 26612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.234.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.253@KRBTEST.COM" created.
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:43.280323 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.267142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:34163 (local address 127.0.0.1:58672)
0619 18:49:43.267721 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.267759 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:43.268660 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:43.269357 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:43.269362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:43.269810 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:43.270058 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.270072 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.271210 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.271213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.271999 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.272011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.272667 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.272670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.272782 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.273382 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:43.273399 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:43.275020 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:43.277192 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.277201 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.277211 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.277497 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.277868 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.277870 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.277872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.277999 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.278825 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:43.278832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:43.278978 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 18:49:43.279134 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 18:49:43.279484 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"spinlock_wait_cycles":31616,"thread_start_us":137,"threads_started":1}
I20260619 18:49:43.280326 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.267141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.12.234.254:34163)
0619 18:49:43.267585 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.267604 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.267922 (+ 318us) server_negotiation.cc:407] Connection header received
0619 18:49:43.268964 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.268968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.269167 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.269335 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.270224 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.271093 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.272111 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.272519 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.272668 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.275174 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.275181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.275193 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.275215 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.277049 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.277611 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.277618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.277620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.277771 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.278086 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.278090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.278092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.278683 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.278833 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.279239 (+ 406us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.279568 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":300,"thread_start_us":125,"threads_started":1}
I20260619 18:49:43.282939 26612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:49:43.283686 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.283400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:40401 (local address 127.0.0.1:41086)
0619 18:49:43.283524 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.283611 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 18:49:43.283877 26563 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
I20260619 18:49:43.284721 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.284522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:40401 (local address 127.0.0.1:41092)
0619 18:49:43.284622 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.284666 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 18:49:43.284842 26612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.253:40401: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab.
Entry for principal kudu/127.12.234.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab.
I20260619 18:49:43.289170 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.253:40401
--webserver_interface=127.12.234.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab
--principal=kudu/127.12.234.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:34163,127.12.234.253:40401,127.12.234.252:43699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:49:43.334126 26612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } attempt: 1
I20260619 18:49:43.335091 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.334712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:40401 (local address 127.0.0.1:41098)
0619 18:49:43.334897 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.334979 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 18:49:43.335948 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.335708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:40401 (local address 127.0.0.1:41100)
0619 18:49:43.335815 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.335872 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 18:49:43.336110 26612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.253:40401: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
W20260619 18:49:43.401453 26619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:43.401748 26619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:43.401788 26619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 18:49:43.404274 26612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } attempt: 2
I20260619 18:49:43.405133 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.404811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:40401 (local address 127.0.0.1:41104)
0619 18:49:43.404966 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.405044 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 18:49:43.405802 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.405605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:40401 (local address 127.0.0.1:41116)
0619 18:49:43.405694 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.405740 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 18:49:43.405907 26612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.253:40401: Network error: Client connection negotiation failed: client connection to 127.12.234.253:40401: connect: Connection refused (error 111)
W20260619 18:49:43.406260 26619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:49:43.406308 26619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:43.406339 26619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:43.406371 26619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:49:43.406392 26619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:49:43.412184 26619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:34163,127.12.234.253:40401,127.12.234.252:43699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab
--principal=kudu/127.12.234.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.253:40401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.234.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:43.413605 26619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:43.414795 26619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:43.423422 26624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:43.423424 26627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:43.423424 26625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:43.423707 26619 server_base.cc:1034] running on GCE node
I20260619 18:49:43.424345 26619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:43.425563 26619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:43.426724 26619 hybrid_clock.cc:648] HybridClock initialized: now 1781894983426703 us; error 33 us; skew 500 ppm
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:43.429466 26619 init.cc:377] Logged in from keytab as kudu/127.12.234.253@KRBTEST.COM (short username kudu)
I20260619 18:49:43.430374 26619 webserver.cc:460] Webserver started at http://127.12.234.253:37185/ using document root <none> and password file <none>
I20260619 18:49:43.430897 26619 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:43.430953 26619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:43.431123 26619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:49:43.433168 26619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/data/instance:
uuid: "cd578521addf43919326f3cad7ea686b"
format_stamp: "Formatted at 2026-06-19 18:49:43 on dist-test-slave-fljv"
server_key: "32827bc35ce485867f41e007c9727726"
server_key_iv: "f8006d08f4c8c15a25a78cd42af807e4"
server_key_version: "encryptionkey@0"
I20260619 18:49:43.433701 26619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/wal/instance:
uuid: "cd578521addf43919326f3cad7ea686b"
format_stamp: "Formatted at 2026-06-19 18:49:43 on dist-test-slave-fljv"
server_key: "32827bc35ce485867f41e007c9727726"
server_key_iv: "f8006d08f4c8c15a25a78cd42af807e4"
server_key_version: "encryptionkey@0"
I20260619 18:49:43.437542 26619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 18:49:43.440052 26634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:43.441318 26619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 18:49:43.441440 26619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/wal
uuid: "cd578521addf43919326f3cad7ea686b"
format_stamp: "Formatted at 2026-06-19 18:49:43 on dist-test-slave-fljv"
server_key: "32827bc35ce485867f41e007c9727726"
server_key_iv: "f8006d08f4c8c15a25a78cd42af807e4"
server_key_version: "encryptionkey@0"
I20260619 18:49:43.441587 26619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:43.464135 26619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:43.467219 26619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:43.467393 26619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:43.474079 26619 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.253:40401
I20260619 18:49:43.474362 26686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.253:40401 every 8 connection(s)
I20260619 18:49:43.475103 26619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
I20260619 18:49:43.475600 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 26619
I20260619 18:49:43.475744 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-1/wal/instance
I20260619 18:49:43.476019 13227 external_mini_cluster.cc:1428] Setting key 18a851e976ceafac556bca2de3585d0c
I20260619 18:49:43.477982 26687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:43.481805 26687 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.234.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.252@KRBTEST.COM" created.
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:43.495440 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.484049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58680 (local address 127.12.234.254:34163)
0619 18:49:43.484215 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.484220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.485106 (+ 886us) server_negotiation.cc:407] Connection header received
0619 18:49:43.486010 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.486013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.486071 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.486142 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.487362 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.487882 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.489068 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.489315 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.489408 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.492021 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.492025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.492027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.492054 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.493345 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.493844 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.493847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.493848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.493928 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.494306 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.494309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.494310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.494508 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.494595 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.495140 (+ 545us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.495275 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 18:49:43.496263 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.484438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:34163 (local address 127.0.0.1:58680)
0619 18:49:43.484927 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.484995 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:43.485797 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:43.486345 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:43.486353 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:43.486704 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:43.487214 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.487228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.488007 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.488010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.488936 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.488947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.489465 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.489469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.489641 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.490260 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:43.490276 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:43.491794 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:43.493456 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.493462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.493478 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.493737 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.494028 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.494030 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.494031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.494208 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.494606 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:43.494614 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:43.494880 (+ 266us) client_negotiation.cc:769] Sending connection context
0619 18:49:43.495051 (+ 171us) client_negotiation.cc:240] Negotiation successful
0619 18:49:43.495401 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":120,"threads_started":1}
I20260619 18:49:43.497049 26612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } attempt: 3
I20260619 18:49:43.497876 26687 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab.
Entry for principal kudu/127.12.234.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab.
I20260619 18:49:43.507022 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.252:43699
--webserver_interface=127.12.234.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab
--principal=kudu/127.12.234.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:34163,127.12.234.253:40401,127.12.234.252:43699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:49:43.508373 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.497396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:40401 (local address 127.0.0.1:41128)
0619 18:49:43.497562 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.497576 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:43.497650 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:43.498473 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:43.498477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:43.498641 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:43.498748 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.498753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.499980 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.499984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.501189 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.501199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.501747 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.501751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.501867 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.502517 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:43.502533 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:43.503918 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:43.506649 (+ 2731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.506653 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.506655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.506905 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.507374 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.507377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.507379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.507473 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.507853 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:43.507856 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:43.507959 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 18:49:43.508053 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 18:49:43.508211 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260619 18:49:43.508356 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.498384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:40401 (local address 127.0.0.1:41140)
0619 18:49:43.498552 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.498566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:43.498638 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:43.499051 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:43.499054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:43.499208 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:43.499329 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.499336 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.500115 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.500118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.500957 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.500966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.501747 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.501751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.501907 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.502555 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:43.502567 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:43.504459 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:43.506685 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.506688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.506690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.506901 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.507258 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.507261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.507263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.507349 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.507867 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:43.507870 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:43.507940 (+ 70us) client_negotiation.cc:769] Sending connection context
0619 18:49:43.508009 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 18:49:43.508155 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":2}
I20260619 18:49:43.508526 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.497549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.12.234.253:40401)
0619 18:49:43.498018 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.498030 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.498051 (+ 21us) server_negotiation.cc:407] Connection header received
0619 18:49:43.498089 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.498094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.498313 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.498867 (+ 554us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.498964 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.499873 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.501309 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.501625 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.501770 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.504084 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.504098 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.504116 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.504146 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.506532 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.507213 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.507216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.507217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.507288 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.507556 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.507559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.507561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.507750 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.507879 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.508169 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.508315 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":377,"spinlock_wait_cycles":5248,"thread_start_us":139,"threads_started":1}
I20260619 18:49:43.508956 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.498433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.12.234.253:40401)
0619 18:49:43.498860 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.498876 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.498888 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:49:43.498914 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.498916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.498948 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.499013 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.499424 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.500020 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.501075 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.501625 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.501765 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.504606 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.504609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.504612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.504634 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.506563 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.507012 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.507019 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.507020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.507112 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.507421 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.507423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.507425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.507758 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.508684 (+ 926us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.508741 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.508823 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":365,"spinlock_wait_cycles":1280,"thread_start_us":80,"threads_started":1}
I20260619 18:49:43.510938 26687 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:49:43.511116 26612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:49:43.511636 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.511404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59326)
0619 18:49:43.511500 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.511573 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 18:49:43.511866 26637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
I20260619 18:49:43.512429 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.511570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59334)
0619 18:49:43.511691 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.512349 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 18:49:43.512763 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.512520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59346)
0619 18:49:43.512659 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.512705 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260619 18:49:43.512948 26687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:43699: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
I20260619 18:49:43.513348 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.512969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59352)
0619 18:49:43.513226 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.513277 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
W20260619 18:49:43.513572 26612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:43699: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
I20260619 18:49:43.544739 26612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } attempt: 1
I20260619 18:49:43.545603 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.545257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59354)
0619 18:49:43.545420 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.545497 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 18:49:43.546423 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.546091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59370)
0619 18:49:43.546300 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.546355 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260619 18:49:43.546566 26612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:43699: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
I20260619 18:49:43.562224 26687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } attempt: 1
I20260619 18:49:43.563108 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.562760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59372)
0619 18:49:43.562927 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.563007 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 18:49:43.563853 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.563640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59382)
0619 18:49:43.563736 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.563788 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 18:49:43.564000 26687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:43699: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
I20260619 18:49:43.621737 26612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } attempt: 2
W20260619 18:49:43.622282 26694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:43.622591 26694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:43.622651 26694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 18:49:43.622792 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.622395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59398)
0619 18:49:43.622628 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.622702 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260619 18:49:43.623555 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.623328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59406)
0619 18:49:43.623431 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.623481 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 18:49:43.623668 26612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:43699: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
W20260619 18:49:43.627115 26694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:49:43.627185 26694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:43.627205 26694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:43.627221 26694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:49:43.627235 26694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:49:43.632180 26687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } attempt: 2
I20260619 18:49:43.632893 26694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:34163,127.12.234.253:40401,127.12.234.252:43699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab
--principal=kudu/127.12.234.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.252:43699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.234.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:43.633090 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.632746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59412)
0619 18:49:43.632918 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.632996 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 18:49:43.633826 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.633611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59414)
0619 18:49:43.633715 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.633768 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 18:49:43.633960 26687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:43699: Network error: Client connection negotiation failed: client connection to 127.12.234.252:43699: connect: Connection refused (error 111)
I20260619 18:49:43.634328 26694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:43.635375 26694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:43.644068 26701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:43.644102 26700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:43.644076 26703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:43.644426 26694 server_base.cc:1034] running on GCE node
I20260619 18:49:43.645028 26694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:43.646206 26694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:43.647369 26694 hybrid_clock.cc:648] HybridClock initialized: now 1781894983647339 us; error 49 us; skew 500 ppm
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:43.650038 26694 init.cc:377] Logged in from keytab as kudu/127.12.234.252@KRBTEST.COM (short username kudu)
I20260619 18:49:43.650971 26694 webserver.cc:460] Webserver started at http://127.12.234.252:32813/ using document root <none> and password file <none>
I20260619 18:49:43.651494 26694 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:43.651551 26694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:43.651721 26694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:49:43.653748 26694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/data/instance:
uuid: "97834d0f2ff94fe982c9b6be40a1e49a"
format_stamp: "Formatted at 2026-06-19 18:49:43 on dist-test-slave-fljv"
server_key: "3c1f9abf9517dbc5f1ba0b13dafb07b4"
server_key_iv: "13db4e9cc81d3b2a1bb1b9d2e9e2dcf5"
server_key_version: "encryptionkey@0"
I20260619 18:49:43.654312 26694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/wal/instance:
uuid: "97834d0f2ff94fe982c9b6be40a1e49a"
format_stamp: "Formatted at 2026-06-19 18:49:43 on dist-test-slave-fljv"
server_key: "3c1f9abf9517dbc5f1ba0b13dafb07b4"
server_key_iv: "13db4e9cc81d3b2a1bb1b9d2e9e2dcf5"
server_key_version: "encryptionkey@0"
I20260619 18:49:43.657891 26694 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260619 18:49:43.660233 26710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:43.661463 26694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:49:43.661592 26694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/wal
uuid: "97834d0f2ff94fe982c9b6be40a1e49a"
format_stamp: "Formatted at 2026-06-19 18:49:43 on dist-test-slave-fljv"
server_key: "3c1f9abf9517dbc5f1ba0b13dafb07b4"
server_key_iv: "13db4e9cc81d3b2a1bb1b9d2e9e2dcf5"
server_key_version: "encryptionkey@0"
I20260619 18:49:43.661700 26694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:43.691270 26694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:43.694221 26694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:43.694399 26694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:43.701177 26694 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.252:43699
I20260619 18:49:43.701422 26762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.252:43699 every 8 connection(s)
I20260619 18:49:43.702149 26694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
I20260619 18:49:43.704969 26763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:43.705406 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 26694
I20260619 18:49:43.705521 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781894826428600-13227-0/minicluster-data/master-2/wal/instance
I20260619 18:49:43.705807 13227 external_mini_cluster.cc:1428] Setting key 1635b095bf3df1efdb902139f0d12d9e
I20260619 18:49:43.709748 26763 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:43.720664 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.707182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59430 (local address 127.12.234.252:43699)
0619 18:49:43.707602 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.707616 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.707652 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:49:43.708353 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.708373 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.708733 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.709072 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.709862 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.710908 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.711883 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.712460 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.712582 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.715337 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.715343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.715354 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.715384 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.717314 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.717828 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.717834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.717841 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.717982 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.718286 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.718289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.718290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.718816 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.718988 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.719321 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.719651 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":123,"threads_started":1}
I20260619 18:49:43.720732 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.711558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58694 (local address 127.12.234.254:34163)
0619 18:49:43.711701 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.711705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.711999 (+ 294us) server_negotiation.cc:407] Connection header received
0619 18:49:43.712130 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.712133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.712176 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.712239 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.713012 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.713753 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.714956 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.715179 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.715268 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.717509 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.717532 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.717535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.717567 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.719082 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.719537 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.719541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.719543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.719624 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.719866 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.719868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.719869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.720054 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.720122 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.720460 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.720573 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 18:49:43.720695 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.711520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:34163 (local address 127.0.0.1:58694)
0619 18:49:43.711890 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.711930 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:43.712045 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:43.712300 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:43.712307 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:43.712739 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:43.712906 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.712915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.713876 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.713879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.714828 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.714839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.715324 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.715329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.715426 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.715908 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:43.715920 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:43.717360 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:43.719182 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.719188 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.719194 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.719439 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.719703 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.719706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.719708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.719789 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.720137 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:43.720143 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:43.720255 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 18:49:43.720392 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 18:49:43.720554 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":92,"threads_started":1}
I20260619 18:49:43.722676 26763 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:49:43.722805 26612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } attempt: 3
I20260619 18:49:43.725111 26687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } attempt: 3
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
I20260619 18:49:43.732654 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.723316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:40401 (local address 127.0.0.1:41148)
0619 18:49:43.723823 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.723836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:43.723911 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:43.724230 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:43.724233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:43.724378 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:43.724492 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.724500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.725452 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.725456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.726332 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.726342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.727028 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.727031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.727134 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.727521 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:43.727535 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:43.728931 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:43.730819 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.730822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.730824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.731150 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.731537 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.731540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.731542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.731631 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.732113 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:43.732116 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:43.732212 (+ 96us) client_negotiation.cc:769] Sending connection context
0619 18:49:43.732322 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 18:49:43.732453 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":1}
I20260619 18:49:43.732667 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.723427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.12.234.253:40401)
0619 18:49:43.723559 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.723563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.723882 (+ 319us) server_negotiation.cc:407] Connection header received
0619 18:49:43.724071 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.724074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.724129 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.724202 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.724610 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.725343 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.726460 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.726808 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.726885 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.729076 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.729079 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.729081 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.729103 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.730713 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.731264 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.731268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.731270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.731360 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.731738 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.731742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.731744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.731966 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.732041 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.732375 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.732503 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 18:49:43.733964 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.723166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59436)
0619 18:49:43.723286 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.723300 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:43.723374 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:43.723722 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:43.723726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:43.723897 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:43.724007 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.724023 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.724906 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.724910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.725662 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.725672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.726306 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.726310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.726395 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.726877 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:43.726892 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:43.728931 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:43.731667 (+ 2736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.731670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.731672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.731978 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.732258 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.732261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.732263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.732349 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.733485 (+ 1136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:43.733488 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:43.733595 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 18:49:43.733675 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 18:49:43.733804 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 18:49:43.733906 26763 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:49:43.734642 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.723418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.12.234.252:43699)
0619 18:49:43.723513 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.723516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.723528 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:49:43.723558 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.723560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.723598 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.724118 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.724201 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.724783 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.725795 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.726154 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.726618 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.729074 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.729077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.729080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.729103 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.731547 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.732094 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.732097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.732099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.732174 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.732827 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.732831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.732833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.733046 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.734302 (+ 1256us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.734373 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.734498 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":33}
I20260619 18:49:43.735438 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.725616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59452)
0619 18:49:43.725775 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.725788 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:43.725850 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:43.726594 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:43.726597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:43.726773 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:43.726867 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.726873 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.728401 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.728405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.729214 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.729226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.729822 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.729826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.729914 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.730287 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:43.730302 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:43.731667 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:43.733627 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.733630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.733632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.733852 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.734153 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.734156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.734158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.734369 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.734973 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:43.734976 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:43.735077 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 18:49:43.735160 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 18:49:43.735287 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260619 18:49:43.735469 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.726027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59452 (local address 127.12.234.252:43699)
0619 18:49:43.726356 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.726364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.726375 (+ 11us) server_negotiation.cc:407] Connection header received
0619 18:49:43.726404 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.726406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.726443 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.726973 (+ 530us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.727059 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.728259 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.729325 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.729698 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.729802 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.731795 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.731799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.731802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.731825 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.733495 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.733962 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.733965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.733967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.734048 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.734663 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.734666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.734668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.734855 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.734975 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.735226 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.735323 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":112,"threads_started":1}
I20260619 18:49:43.736007 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.723695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.12.234.254:34163)
0619 18:49:43.723802 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.723806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.723837 (+ 31us) server_negotiation.cc:407] Connection header received
0619 18:49:43.723950 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.723953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.723995 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.724060 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.724828 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.725504 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.726871 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.727165 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.727256 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.730883 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.730887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.730890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.730915 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.732705 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.733367 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.733370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.733372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.733454 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.733760 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.733763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.733765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.733958 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.734448 (+ 490us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.735392 (+ 944us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.735868 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 18:49:43.738375 26612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17: Bootstrap starting.
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
I20260619 18:49:43.741456 26612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17: Neither blocks nor log segments found. Creating new log.
Jun 19 18:49:43 dist-test-slave-fljv krb5kdc[26531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
I20260619 18:49:43.743182 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.734418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:43699 (local address 127.0.0.1:59462)
0619 18:49:43.735519 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:43.735534 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:43.735630 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:43.735920 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:43.735923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:43.736081 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:43.736159 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.736165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.737123 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.737126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.737705 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:43.737713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.738222 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.738226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:43.738300 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.738715 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:43.738728 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:43.740133 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:43.741826 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.741829 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.741831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.742006 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.742299 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:43.742302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:43.742304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:43.742375 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:43.742704 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:43.742707 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:43.742785 (+ 78us) client_negotiation.cc:769] Sending connection context
0619 18:49:43.742865 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 18:49:43.743007 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036}
I20260619 18:49:43.743184 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.734606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59462 (local address 127.12.234.252:43699)
0619 18:49:43.734781 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.734785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.735578 (+ 793us) server_negotiation.cc:407] Connection header received
0619 18:49:43.735724 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.735727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.735769 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.735837 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.736517 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.737023 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.737802 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.738095 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.738201 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.740262 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.740266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.740268 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.740294 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.741737 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.742083 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.742086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.742088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.742158 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.742445 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.742448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.742450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.742624 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.742704 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.742915 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.743061 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":99,"threads_started":1}
I20260619 18:49:43.745076 26687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b: Bootstrap starting.
I20260619 18:49:43.746294 26763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a: Bootstrap starting.
I20260619 18:49:43.746614 26612 log.cc:824] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:43.747285 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.737296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.12.234.253:40401)
0619 18:49:43.737418 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.737422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.737961 (+ 539us) server_negotiation.cc:407] Connection header received
0619 18:49:43.738047 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.738050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.738095 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.738166 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.738958 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.739531 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.740335 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.740637 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.740731 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.743496 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.743501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.743503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.743531 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.745028 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.745707 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.745711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.745713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.745805 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.746093 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.746096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.746097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.746344 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.746524 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.747010 (+ 486us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.747142 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 18:49:43.747970 26687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:43.748792 26763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:43.748813 26687 log.cc:824] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:43.749522 26763 log.cc:824] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:43.751065 26687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b: No bootstrap required, opened a new log
I20260619 18:49:43.751098 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:49:43.751569 26763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a: No bootstrap required, opened a new log
I20260619 18:49:43.755249 26687 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } }
I20260619 18:49:43.755250 26763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } }
I20260619 18:49:43.755486 26763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:43.755486 26687 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:43.755535 26763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97834d0f2ff94fe982c9b6be40a1e49a, State: Initialized, Role: FOLLOWER
I20260619 18:49:43.755549 26687 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd578521addf43919326f3cad7ea686b, State: Initialized, Role: FOLLOWER
I20260619 18:49:43.756031 26687 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } }
I20260619 18:49:43.756035 26763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } }
I20260619 18:49:43.757455 26763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:49:43.757455 26687 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:49:43.762066 26612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17: No bootstrap required, opened a new log
W20260619 18:49:43.763281 26803 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:49:43.763363 26803 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:49:43.764330 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.752998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.12.234.252:43699)
0619 18:49:43.753114 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.753118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.753959 (+ 841us) server_negotiation.cc:407] Connection header received
0619 18:49:43.754064 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.754067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.754118 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.754220 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.755108 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.755874 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.757300 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.757606 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.757674 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.759781 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.759786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.759789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.759815 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.761764 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.762964 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.762968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.762971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.763058 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.763362 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.763365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.763367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.763597 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.763717 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.764051 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.764174 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 18:49:43.766474 26612 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } }
I20260619 18:49:43.766731 26612 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:43.766786 26612 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f84b5b64fe483aba053bc57ea8ab17, State: Initialized, Role: FOLLOWER
I20260619 18:49:43.767323 26612 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } }
I20260619 18:49:43.768690 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.753142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58720 (local address 127.12.234.254:34163)
0619 18:49:43.753247 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.753251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.754255 (+ 1004us) server_negotiation.cc:407] Connection header received
0619 18:49:43.754418 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.754422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.754483 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.754534 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.755316 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.756029 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.757392 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.757712 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.757778 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.760262 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.760266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.760269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.760296 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.762481 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.766384 (+ 3903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.766388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.766390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.766504 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.766792 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.766796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.766797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.767023 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.767145 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.768371 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.768526 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:49:43.768842 26612 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:49:43.771648 26780 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.771793 26780 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:49:43.771898 26780 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:49:43.763319 26619 master_runner.cc:186] Error getting master registration for 127.12.234.254:34163: OK, instance_id { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" instance_seqno: 1781894983254759 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 18:49:43.772068 26619 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 18:49:43.772198 26780 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } }
I20260619 18:49:43.767884 26694 master_runner.cc:186] Error getting master registration for 127.12.234.254:34163: OK, instance_id { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" instance_seqno: 1781894983254759 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 18:49:43.772874 26694 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 18:49:43.773363 26780 leader_election.cc:288] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f84b5b64fe483aba053bc57ea8ab17 (127.12.234.254:34163), 97834d0f2ff94fe982c9b6be40a1e49a (127.12.234.252:43699)
I20260619 18:49:43.773666 26806 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.773777 26806 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:49:43.773802 26737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd578521addf43919326f3cad7ea686b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" is_pre_election: true
I20260619 18:49:43.774103 26737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd578521addf43919326f3cad7ea686b in term 0.
I20260619 18:49:43.774524 26781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.774544 26637 leader_election.cc:302] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97834d0f2ff94fe982c9b6be40a1e49a, cd578521addf43919326f3cad7ea686b; no voters:
I20260619 18:49:43.774657 26781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:49:43.775178 26586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd578521addf43919326f3cad7ea686b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" is_pre_election: true
I20260619 18:49:43.775460 26586 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd578521addf43919326f3cad7ea686b in term 0.
I20260619 18:49:43.776654 26805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:49:43.776808 26805 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:49:43.776909 26805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 0 FOLLOWER]: Advancing to term 1
W20260619 18:49:43.777904 26797 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:49:43.777972 26797 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:49:43.778003 26544 master_runner.cc:263] No leader master found from master a5f84b5b64fe483aba053bc57ea8ab17
I20260619 18:49:43.778054 26544 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 18:49:43.779336 26805 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } }
I20260619 18:49:43.780049 26586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd578521addf43919326f3cad7ea686b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f84b5b64fe483aba053bc57ea8ab17"
I20260619 18:49:43.780171 26586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:43.781431 26586 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd578521addf43919326f3cad7ea686b in term 1.
I20260619 18:49:43.781661 26737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd578521addf43919326f3cad7ea686b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97834d0f2ff94fe982c9b6be40a1e49a"
I20260619 18:49:43.781773 26636 leader_election.cc:302] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5f84b5b64fe483aba053bc57ea8ab17, cd578521addf43919326f3cad7ea686b; no voters:
I20260619 18:49:43.781780 26737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:43.781925 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:43.753674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41152 (local address 127.12.234.253:40401)
0619 18:49:43.753777 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:43.753782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:43.756566 (+ 2784us) server_negotiation.cc:407] Connection header received
0619 18:49:43.758368 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:43.758371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:43.758423 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:43.758485 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:43.767446 (+ 8961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.768180 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.770154 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:43.770529 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:43.770613 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:43.778342 (+ 7729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:43.778346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:43.778349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:43.778376 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:43.780060 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.780576 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.780580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.780582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.780664 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:43.780940 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:43.780943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:43.780945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:43.781169 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:43.781276 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:43.781661 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 18:49:43.781782 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:49:43.782161 26780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:49:43.782388 26780 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [term 1 LEADER]: Becoming Leader. State: Replica: cd578521addf43919326f3cad7ea686b, State: Running, Role: LEADER
I20260619 18:49:43.782742 26780 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } }
I20260619 18:49:43.783052 26737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd578521addf43919326f3cad7ea686b in term 1.
I20260619 18:49:43.783561 26805 leader_election.cc:288] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [CANDIDATE]: Term 1 election: Requested vote from peers a5f84b5b64fe483aba053bc57ea8ab17 (127.12.234.254:34163), 97834d0f2ff94fe982c9b6be40a1e49a (127.12.234.252:43699)
W20260619 18:49:43.783756 26817 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:49:43.783818 26817 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:49:43.785300 26805 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd578521addf43919326f3cad7ea686b. Latest consensus state: current_term: 1 leader_uuid: "cd578521addf43919326f3cad7ea686b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.785425 26805 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:43.786501 26822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:49:43.787461 26822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:49:43.800464 26737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [term 1 FOLLOWER]: Refusing update from remote peer cd578521addf43919326f3cad7ea686b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:49:43.801252 26586 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [term 1 FOLLOWER]: Refusing update from remote peer cd578521addf43919326f3cad7ea686b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:49:43.801187 26820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [LEADER]: Connected to new peer: Peer: permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:49:43.805698 26781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd578521addf43919326f3cad7ea686b. Latest consensus state: current_term: 1 leader_uuid: "cd578521addf43919326f3cad7ea686b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.805832 26781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:49:43.806090 26820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:49:43.810678 26806 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd578521addf43919326f3cad7ea686b. Latest consensus state: current_term: 1 leader_uuid: "cd578521addf43919326f3cad7ea686b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.810823 26806 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:49:43.812753 26820 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd578521addf43919326f3cad7ea686b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.812904 26820 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:43.813025 26823 mvcc.cc:205] Tried to move back new op lower bound from 7298641853637951488 to 7298641853577515008. Current Snapshot: MvccSnapshot[applied={T|T < 7298641853637951488}]
I20260619 18:49:43.814064 26781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd578521addf43919326f3cad7ea686b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.814214 26781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:49:43.816068 26824 mvcc.cc:205] Tried to move back new op lower bound from 7298641853637951488 to 7298641853577515008. Current Snapshot: MvccSnapshot[applied={T|T < 7298641853637951488}]
I20260619 18:49:43.822683 26822 catalog_manager.cc:1347] Generated new cluster ID: 9a124abac41144c98154b619af75da2a
I20260619 18:49:43.822777 26822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:49:43.823764 26780 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd578521addf43919326f3cad7ea686b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.823902 26780 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:43.825093 26806 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd578521addf43919326f3cad7ea686b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f84b5b64fe483aba053bc57ea8ab17" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34163 } } peers { permanent_uuid: "cd578521addf43919326f3cad7ea686b" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 40401 } } peers { permanent_uuid: "97834d0f2ff94fe982c9b6be40a1e49a" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 43699 } } }
I20260619 18:49:43.825219 26806 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:49:43.825711 26826 mvcc.cc:205] Tried to move back new op lower bound from 7298641853637951488 to 7298641853577515008. Current Snapshot: MvccSnapshot[applied={T|T < 7298641853637951488}]
I20260619 18:49:43.838133 26822 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:49:43.839496 26822 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:49:43.862519 26822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd578521addf43919326f3cad7ea686b: Generated new TSK 0
I20260619 18:49:43.863551 26822 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:49:44.779361 26797 catalog_manager.cc:1259] Loaded cluster ID: 9a124abac41144c98154b619af75da2a
I20260619 18:49:44.779450 26797 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a: loading cluster ID for follower catalog manager: success
I20260619 18:49:44.781118 26797 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a: acquiring CA information for follower catalog manager: success
I20260619 18:49:44.782338 26797 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 97834d0f2ff94fe982c9b6be40a1e49a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 18:49:44.785064 26817 catalog_manager.cc:1259] Loaded cluster ID: 9a124abac41144c98154b619af75da2a
I20260619 18:49:44.785141 26817 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17: loading cluster ID for follower catalog manager: success
I20260619 18:49:44.786832 26817 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17: acquiring CA information for follower catalog manager: success
I20260619 18:49:44.788041 26817 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5f84b5b64fe483aba053bc57ea8ab17: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 18:49:44.926787 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 26544
I20260619 18:49:44.932456 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 26619
I20260619 18:49:44.937687 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 26694
2026-06-19T18:49:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2958 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:49:44 dist-test-slave-fljv krb5kdc[26837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:49:44 dist-test-slave-fljv krb5kdc[26837](info): set up 2 sockets
Jun 19 18:49:44 dist-test-slave-fljv krb5kdc[26837](info): commencing operation
krb5kdc: starting...
W20260619 18:49:45.890256 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:49:45 dist-test-slave-fljv krb5kdc[26837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:49:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:49:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:49:46.023106 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:37271
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:40887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:37271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:46.133422 26850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:46.133740 26850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:46.133798 26850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:46.137931 26850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:49:46.138017 26850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:46.138044 26850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:46.138064 26850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:49:46.138084 26850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:49:46.143569 26850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:40887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:37271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:37271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:46.144745 26850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:46.145797 26850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:49:46.154668 26850 server_base.cc:1034] running on GCE node
W20260619 18:49:46.154629 26856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:46.154642 26855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:46.154623 26858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:46.155529 26850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:46.156683 26850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:46.157867 26850 hybrid_clock.cc:648] HybridClock initialized: now 1781894986157846 us; error 37 us; skew 500 ppm
Jun 19 18:49:46 dist-test-slave-fljv krb5kdc[26837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894986, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:46.160769 26850 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:49:46.161734 26850 webserver.cc:460] Webserver started at http://127.12.234.254:43369/ using document root <none> and password file <none>
I20260619 18:49:46.162321 26850 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:46.162391 26850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:46.162573 26850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:49:46.164659 26850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "aba008fd0cc140eb8ccfaa59bf22fe45"
format_stamp: "Formatted at 2026-06-19 18:49:46 on dist-test-slave-fljv"
server_key: "65a629f45191910fe52b952c54c0a6c7"
server_key_iv: "78c6bebde7ffe326fb2126094126d8a7"
server_key_version: "encryptionkey@0"
I20260619 18:49:46.165191 26850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "aba008fd0cc140eb8ccfaa59bf22fe45"
format_stamp: "Formatted at 2026-06-19 18:49:46 on dist-test-slave-fljv"
server_key: "65a629f45191910fe52b952c54c0a6c7"
server_key_iv: "78c6bebde7ffe326fb2126094126d8a7"
server_key_version: "encryptionkey@0"
I20260619 18:49:46.169155 26850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:49:46.171713 26865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:46.172968 26850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 18:49:46.173080 26850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "aba008fd0cc140eb8ccfaa59bf22fe45"
format_stamp: "Formatted at 2026-06-19 18:49:46 on dist-test-slave-fljv"
server_key: "65a629f45191910fe52b952c54c0a6c7"
server_key_iv: "78c6bebde7ffe326fb2126094126d8a7"
server_key_version: "encryptionkey@0"
I20260619 18:49:46.173184 26850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:46.187168 26850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:46.190165 26850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:46.190354 26850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:46.197109 26850 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:37271
I20260619 18:49:46.197139 26917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:37271 every 8 connection(s)
I20260619 18:49:46.198105 26850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:49:46.199499 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 26850
I20260619 18:49:46.199592 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:49:46.199864 13227 external_mini_cluster.cc:1428] Setting key 4f8c03de7bbbbb25cf01bf067eea8ced
I20260619 18:49:46.201246 26918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:46.207000 26918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45: Bootstrap starting.
Jun 19 18:49:46 dist-test-slave-fljv krb5kdc[26837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:46.209374 26918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:46.210199 26918 log.cc:824] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:46.212133 26918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45: No bootstrap required, opened a new log
I20260619 18:49:46.213212 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:46.201215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.12.234.254:37271)
0619 18:49:46.201654 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:46.201670 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:46.201707 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:49:46.202417 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:46.202434 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:46.202778 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:46.203094 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:46.203883 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:46.204867 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:46.205730 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:46.206250 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:46.206397 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:46.208782 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:46.208787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:46.208801 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:46.208830 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:46.210504 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:46.210906 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:46.210913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:46.210918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:46.211029 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:46.211293 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:46.211296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:46.211297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:46.211698 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:46.211863 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:46.212172 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 18:49:46.212485 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":138,"threads_started":1}
I20260619 18:49:46.215788 26918 raft_consensus.cc:348] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba008fd0cc140eb8ccfaa59bf22fe45" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37271 } }
I20260619 18:49:46.215996 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:49:46.216041 26918 raft_consensus.cc:374] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:46.216095 26918 raft_consensus.cc:729] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba008fd0cc140eb8ccfaa59bf22fe45, State: Initialized, Role: FOLLOWER
I20260619 18:49:46.216562 26918 consensus_queue.cc:260] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba008fd0cc140eb8ccfaa59bf22fe45" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37271 } }
I20260619 18:49:46.216734 26918 raft_consensus.cc:388] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:49:46.216802 26918 raft_consensus.cc:482] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:49:46.216902 26918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:46.217844 26918 raft_consensus.cc:504] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba008fd0cc140eb8ccfaa59bf22fe45" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37271 } }
I20260619 18:49:46.218202 26918 leader_election.cc:302] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aba008fd0cc140eb8ccfaa59bf22fe45; no voters:
I20260619 18:49:46.218529 26918 leader_election.cc:288] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:49:46.218647 26923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:49:46.218854 26923 raft_consensus.cc:686] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [term 1 LEADER]: Becoming Leader. State: Replica: aba008fd0cc140eb8ccfaa59bf22fe45, State: Running, Role: LEADER
I20260619 18:49:46.219199 26923 consensus_queue.cc:237] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba008fd0cc140eb8ccfaa59bf22fe45" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37271 } }
I20260619 18:49:46.219712 26918 sys_catalog.cc:564] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:49:46.220769 26925 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aba008fd0cc140eb8ccfaa59bf22fe45. Latest consensus state: current_term: 1 leader_uuid: "aba008fd0cc140eb8ccfaa59bf22fe45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba008fd0cc140eb8ccfaa59bf22fe45" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37271 } } }
I20260619 18:49:46.220912 26925 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:46.221197 26924 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aba008fd0cc140eb8ccfaa59bf22fe45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba008fd0cc140eb8ccfaa59bf22fe45" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37271 } } }
I20260619 18:49:46.221289 26924 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:46.221318 26932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:49:46.224181 26932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:49:46.229744 26932 catalog_manager.cc:1347] Generated new cluster ID: c070b6cf67d04828a6cec7d1cf6ec667
I20260619 18:49:46.229832 26932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:49:46.240273 26932 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:49:46.241223 26932 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:49:46.246995 26932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aba008fd0cc140eb8ccfaa59bf22fe45: Generated new TSK 0
I20260619 18:49:46.247680 26932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:49:46.271834 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 26850
2026-06-19T18:49:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1332 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:49:46 dist-test-slave-fljv krb5kdc[26945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:49:46 dist-test-slave-fljv krb5kdc[26945](info): set up 2 sockets
Jun 19 18:49:46 dist-test-slave-fljv krb5kdc[26945](info): commencing operation
krb5kdc: starting...
W20260619 18:49:47.226306 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:49:47 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:49:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:49:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:49:47.349084 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46765
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:46765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:47.461275 26958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:47.461609 26958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:47.461660 26958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:47.465891 26958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:49:47.465965 26958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:47.465986 26958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:47.466003 26958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:49:47.466018 26958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:49:47.471716 26958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:46765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:47.472846 26958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:47.473928 26958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:49:47.482757 26958 server_base.cc:1034] running on GCE node
W20260619 18:49:47.482739 26966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:47.482745 26963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:47.482931 26964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:47.483659 26958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:47.484803 26958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:47.485976 26958 hybrid_clock.cc:648] HybridClock initialized: now 1781894987485955 us; error 34 us; skew 500 ppm
Jun 19 18:49:47 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:47.493014 26958 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:49:47.493990 26958 webserver.cc:460] Webserver started at http://127.12.234.254:37727/ using document root <none> and password file <none>
I20260619 18:49:47.494638 26958 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:47.494714 26958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:47.494920 26958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:49:47.496986 26958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "ca10f8244b5e4b6a9d775ee293671288"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "91d25e0688cd48c9dc1d0e6b8c2b1a4c"
server_key_iv: "1039074227a0a21ac206e0f2b1afadff"
server_key_version: "encryptionkey@0"
I20260619 18:49:47.497535 26958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "ca10f8244b5e4b6a9d775ee293671288"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "91d25e0688cd48c9dc1d0e6b8c2b1a4c"
server_key_iv: "1039074227a0a21ac206e0f2b1afadff"
server_key_version: "encryptionkey@0"
I20260619 18:49:47.501296 26958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:49:47.503875 26973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:47.505179 26958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 18:49:47.505302 26958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "ca10f8244b5e4b6a9d775ee293671288"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "91d25e0688cd48c9dc1d0e6b8c2b1a4c"
server_key_iv: "1039074227a0a21ac206e0f2b1afadff"
server_key_version: "encryptionkey@0"
I20260619 18:49:47.505401 26958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:47.530921 26958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:47.534327 26958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:47.534535 26958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:47.541527 26958 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:46765
I20260619 18:49:47.541836 27025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:46765 every 8 connection(s)
I20260619 18:49:47.542644 26958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:49:47.545471 27026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:47.545737 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 26958
I20260619 18:49:47.545874 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:49:47.546236 13227 external_mini_cluster.cc:1428] Setting key bbf8742ca2e762e3f6372441a6013066
I20260619 18:49:47.552838 27026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Bootstrap starting.
Jun 19 18:49:47 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:47.555684 27026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:47.556541 27026 log.cc:824] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:47.558780 27026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: No bootstrap required, opened a new log
I20260619 18:49:47.560995 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:47.547755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36714 (local address 127.12.234.254:46765)
0619 18:49:47.548227 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:47.548244 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:47.548282 (+ 38us) server_negotiation.cc:407] Connection header received
0619 18:49:47.549000 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:47.549020 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:47.549382 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:47.549676 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:47.550633 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:47.551631 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:47.552840 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:47.553408 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:47.553543 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:47.556314 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:47.556323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:47.556341 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:47.556377 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:47.558080 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:47.558603 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:47.558608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:47.558659 (+ 51us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:47.558776 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:47.559060 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:47.559063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:47.559064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:47.559472 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:47.559651 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:47.559926 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 18:49:47.560213 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":133,"threads_started":1}
I20260619 18:49:47.562570 27026 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:47.562799 27026 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:47.562834 27026 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca10f8244b5e4b6a9d775ee293671288, State: Initialized, Role: FOLLOWER
I20260619 18:49:47.563271 27026 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:47.563401 27026 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:49:47.563460 27026 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:49:47.563546 27026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:47.564477 27026 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:47.564797 27026 leader_election.cc:302] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca10f8244b5e4b6a9d775ee293671288; no voters:
I20260619 18:49:47.565128 27026 leader_election.cc:288] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:49:47.565325 27031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:49:47.565804 27031 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 1 LEADER]: Becoming Leader. State: Replica: ca10f8244b5e4b6a9d775ee293671288, State: Running, Role: LEADER
I20260619 18:49:47.566172 27031 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:47.566471 27026 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:49:47.567860 27033 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca10f8244b5e4b6a9d775ee293671288. Latest consensus state: current_term: 1 leader_uuid: "ca10f8244b5e4b6a9d775ee293671288" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } } }
I20260619 18:49:47.567900 27032 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca10f8244b5e4b6a9d775ee293671288" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } } }
I20260619 18:49:47.567994 27033 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:47.567994 27032 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: This master's current role is: LEADER
W20260619 18:49:47.571050 27046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:49:47.571148 27046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:49:47.571198 27044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:49:47.571908 27044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:49:47.577049 27044 catalog_manager.cc:1347] Generated new cluster ID: 63648c854de848c49a3ef21cc9c796b6
I20260619 18:49:47.577133 27044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:49:47.584167 27044 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:49:47.585443 27044 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:49:47.590417 27044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Generated new TSK 0
I20260619 18:49:47.591182 27044 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:49:47.623840 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46765
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:47.736946 27052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:47.737236 27052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:47.737298 27052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:47.741357 27052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:47.741428 27052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:47.741513 27052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:49:47.746928 27052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:47.748117 27052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:47.749181 27052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:47.759016 27058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:47.759016 27057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:47.759187 27052 server_base.cc:1034] running on GCE node
W20260619 18:49:47.759013 27060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:47.759857 27052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:47.760543 27052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:47.761751 27052 hybrid_clock.cc:648] HybridClock initialized: now 1781894987761720 us; error 49 us; skew 500 ppm
Jun 19 18:49:47 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:47.764837 27052 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:49:47.765805 27052 webserver.cc:460] Webserver started at http://127.12.234.193:45003/ using document root <none> and password file <none>
I20260619 18:49:47.766414 27052 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:47.766479 27052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:47.766685 27052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:49:47.768941 27052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "2a5649f03b3e485bb59f4ddccff6834a"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "381461cfdcc55f7101baad2232611fbe"
server_key_iv: "f0b0457ce212862b596f91af252aa48f"
server_key_version: "encryptionkey@0"
I20260619 18:49:47.769495 27052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "2a5649f03b3e485bb59f4ddccff6834a"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "381461cfdcc55f7101baad2232611fbe"
server_key_iv: "f0b0457ce212862b596f91af252aa48f"
server_key_version: "encryptionkey@0"
I20260619 18:49:47.773351 27052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:49:47.775950 27067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:47.777364 27052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:49:47.777493 27052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "2a5649f03b3e485bb59f4ddccff6834a"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "381461cfdcc55f7101baad2232611fbe"
server_key_iv: "f0b0457ce212862b596f91af252aa48f"
server_key_version: "encryptionkey@0"
I20260619 18:49:47.777624 27052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:47.801234 27052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:47.804600 27052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:47.804790 27052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:47.805415 27052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:47.806318 27052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:47.806365 27052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:47.806408 27052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:47.806437 27052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:47.815142 27052 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:32801
I20260619 18:49:47.815163 27180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:32801 every 8 connection(s)
I20260619 18:49:47.816080 27052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:49:47.821089 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 27052
I20260619 18:49:47.821208 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:49:47.821463 13227 external_mini_cluster.cc:1428] Setting key 123e4be5f6ef755b2b908708184b3594
Jun 19 18:49:47 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:49:47.830601 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:47.817917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:52785 (local address 127.12.234.254:46765)
0619 18:49:47.818060 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:47.818065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:47.819020 (+ 955us) server_negotiation.cc:407] Connection header received
0619 18:49:47.820428 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:47.820432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:47.820538 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:47.820623 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:47.822138 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:47.822819 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:47.824119 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:47.824365 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:47.824456 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:47.826940 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:47.826943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:47.826946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:47.826971 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:47.828431 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:47.828947 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:47.828950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:47.828952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:47.829028 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:47.829347 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:47.829351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:47.829353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:47.829541 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:47.829725 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:47.830283 (+ 558us) server_negotiation.cc:299] Negotiation successful
0619 18:49:47.830423 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 18:49:47.831496 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:47.818382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:52785)
0619 18:49:47.818862 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:47.818906 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:47.820199 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:47.820858 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:47.820866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:47.821328 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:47.821977 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:47.821998 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:47.822974 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:47.822978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:47.823984 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:47.823995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:47.824511 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:47.824515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:47.824684 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:47.825266 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:47.825283 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:47.826771 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:47.828544 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:47.828550 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:47.828569 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:47.828850 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:47.829111 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:47.829114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:47.829116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:47.829261 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:47.829728 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:47.829736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:47.830008 (+ 272us) client_negotiation.cc:769] Sending connection context
0619 18:49:47.830232 (+ 224us) client_negotiation.cc:240] Negotiation successful
0619 18:49:47.830594 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":122,"threads_started":1}
I20260619 18:49:47.832899 27181 heartbeater.cc:347] Connected to a master server at 127.12.234.254:46765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:49:47.833179 27181 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:47.833794 27181 heartbeater.cc:511] Master 127.12.234.254:46765 requested a full tablet report, sending...
I20260619 18:49:47.835455 26990 ts_manager.cc:194] Registered new tserver with Master: 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801)
I20260619 18:49:47.836963 26990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:52785
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:49:47.849603 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46765
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:47.963856 27186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:47.964148 27186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:47.964191 27186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:47.968461 27186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:47.968537 27186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:47.968617 27186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:49:47.974117 27186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:47.975260 27186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:47.976289 27186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:47.986028 27191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:47.986028 27192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:47.986030 27194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:47.986634 27186 server_base.cc:1034] running on GCE node
I20260619 18:49:47.987023 27186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:47.987673 27186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:47.988832 27186 hybrid_clock.cc:648] HybridClock initialized: now 1781894987988810 us; error 34 us; skew 500 ppm
Jun 19 18:49:47 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:47.991617 27186 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:49:47.992507 27186 webserver.cc:460] Webserver started at http://127.12.234.194:36313/ using document root <none> and password file <none>
I20260619 18:49:47.993057 27186 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:47.993129 27186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:47.993328 27186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:49:47.995515 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "3d144ccf030c444f95a120f25d3123ae"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "fc40cf871a8a6352f74ba1f56d088090"
server_key_iv: "588b7cddff268a534c08820d07598c2b"
server_key_version: "encryptionkey@0"
I20260619 18:49:47.996039 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "3d144ccf030c444f95a120f25d3123ae"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "fc40cf871a8a6352f74ba1f56d088090"
server_key_iv: "588b7cddff268a534c08820d07598c2b"
server_key_version: "encryptionkey@0"
I20260619 18:49:47.999768 27186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 18:49:48.002224 27201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:48.003562 27186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:49:48.003681 27186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "3d144ccf030c444f95a120f25d3123ae"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "fc40cf871a8a6352f74ba1f56d088090"
server_key_iv: "588b7cddff268a534c08820d07598c2b"
server_key_version: "encryptionkey@0"
I20260619 18:49:48.003786 27186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:48.024041 27186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:48.027565 27186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:48.027769 27186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:48.028429 27186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:48.029327 27186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:48.029376 27186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:48.029429 27186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:48.029459 27186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:48.038822 27186 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:42583
I20260619 18:49:48.038843 27314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:42583 every 8 connection(s)
I20260619 18:49:48.039786 27186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:49:48.046283 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 27186
I20260619 18:49:48.046422 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:49:48.046757 13227 external_mini_cluster.cc:1428] Setting key d66ae5ad30a04978dd618bdf4722aaba
Jun 19 18:49:48 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:48.055903 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.041831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:42115 (local address 127.12.234.254:46765)
0619 18:49:48.042016 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.042019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.043028 (+ 1009us) server_negotiation.cc:407] Connection header received
0619 18:49:48.044459 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.044462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.044522 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.044606 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:48.045859 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.046427 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.047982 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.048326 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.048445 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.051581 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:48.051585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:48.051587 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:48.051614 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:48.053292 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.054077 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.054080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.054082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.054170 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.054660 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.054663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.054665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.054869 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:48.054951 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.055660 (+ 709us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.055780 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:49:48.057159 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.042293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:42115)
0619 18:49:48.042854 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.042901 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.044215 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.044793 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.044802 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.045195 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:48.045706 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.045720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.046558 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.046563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.047845 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.047857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.048523 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.048528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.048813 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.049455 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:48.049477 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:48.051371 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:48.053442 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:48.053450 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:48.053526 (+ 76us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:48.053942 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:48.054287 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:48.054290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:48.054292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:48.054519 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:48.054976 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:48.054984 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:48.055359 (+ 375us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.055602 (+ 243us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.055979 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":123,"threads_started":1}
I20260619 18:49:48.058492 27315 heartbeater.cc:347] Connected to a master server at 127.12.234.254:46765
I20260619 18:49:48.058884 27315 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:48.059449 27315 heartbeater.cc:511] Master 127.12.234.254:46765 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:49:48.060611 26990 ts_manager.cc:194] Registered new tserver with Master: 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583)
I20260619 18:49:48.061352 26990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:42115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:49:48.076800 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46765
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:48.187613 27320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:48.187893 27320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:48.187937 27320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:48.192018 27320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:48.192091 27320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:48.192152 27320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:49:48.197623 27320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:48.198833 27320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:48.199832 27320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:48.209717 27326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:48.209725 27325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:48.209725 27328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:48.210449 27320 server_base.cc:1034] running on GCE node
I20260619 18:49:48.210897 27320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:48.211580 27320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:48.212771 27320 hybrid_clock.cc:648] HybridClock initialized: now 1781894988212750 us; error 42 us; skew 500 ppm
Jun 19 18:49:48 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894988, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:48.215755 27320 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:49:48.216697 27320 webserver.cc:460] Webserver started at http://127.12.234.195:33409/ using document root <none> and password file <none>
I20260619 18:49:48.217252 27320 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:48.217319 27320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:48.217533 27320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:49:48.219935 27320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "4e037a2bcf6a4b93bc4a5da228b98d30"
format_stamp: "Formatted at 2026-06-19 18:49:48 on dist-test-slave-fljv"
server_key: "2b2c61428031eb334206ce9cf2377daa"
server_key_iv: "d00587a7715682580f0299e3736ba852"
server_key_version: "encryptionkey@0"
I20260619 18:49:48.220561 27320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "4e037a2bcf6a4b93bc4a5da228b98d30"
format_stamp: "Formatted at 2026-06-19 18:49:48 on dist-test-slave-fljv"
server_key: "2b2c61428031eb334206ce9cf2377daa"
server_key_iv: "d00587a7715682580f0299e3736ba852"
server_key_version: "encryptionkey@0"
I20260619 18:49:48.224479 27320 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:49:48.227191 27335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:48.228605 27320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:49:48.228725 27320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "4e037a2bcf6a4b93bc4a5da228b98d30"
format_stamp: "Formatted at 2026-06-19 18:49:48 on dist-test-slave-fljv"
server_key: "2b2c61428031eb334206ce9cf2377daa"
server_key_iv: "d00587a7715682580f0299e3736ba852"
server_key_version: "encryptionkey@0"
I20260619 18:49:48.228858 27320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:48.261054 27320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:48.264550 27320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:48.264747 27320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:48.265378 27320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:48.266311 27320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:48.266357 27320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:48.266397 27320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:48.266425 27320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:48.275435 27320 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:37405
I20260619 18:49:48.275456 27448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:37405 every 8 connection(s)
I20260619 18:49:48.276441 27320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:49:48.283532 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 27320
I20260619 18:49:48.283650 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:49:48.283917 13227 external_mini_cluster.cc:1428] Setting key 01064b68aa1bc119682ce4b6d81d5780
Jun 19 18:49:48 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894988, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:48.290386 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.278322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:38743 (local address 127.12.234.254:46765)
0619 18:49:48.278513 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.278517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.279404 (+ 887us) server_negotiation.cc:407] Connection header received
0619 18:49:48.280841 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.280844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.280896 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.280974 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:48.282330 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.282867 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.284166 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.284462 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.284557 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.287287 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:48.287291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:48.287294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:48.287323 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:48.288509 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.288985 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.288987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.288989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.289090 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.289402 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.289404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.289405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.289576 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:48.289651 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.290124 (+ 473us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.290266 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 18:49:48.291229 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.278753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:38743)
0619 18:49:48.279236 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.279282 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.280563 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.281156 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.281164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.281639 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:48.282162 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.282202 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.282990 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.282993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.284044 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.284054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.284620 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.284625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.284820 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.285446 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:48.285465 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:48.287133 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:48.288620 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:48.288625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:48.288638 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:48.288891 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:48.289178 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:48.289181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:48.289183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:48.289329 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:48.289656 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:48.289661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:48.289907 (+ 246us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.290059 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.290389 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":128,"threads_started":1}
I20260619 18:49:48.292315 27449 heartbeater.cc:347] Connected to a master server at 127.12.234.254:46765
I20260619 18:49:48.292541 27449 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:48.293088 27449 heartbeater.cc:511] Master 127.12.234.254:46765 requested a full tablet report, sending...
I20260619 18:49:48.294075 26990 ts_manager.cc:194] Registered new tserver with Master: 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405)
I20260619 18:49:48.294654 26990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:38743
I20260619 18:49:48.297796 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:49:48.307024 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.299721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36728 (local address 127.12.234.254:46765)
0619 18:49:48.299848 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.299852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.300034 (+ 182us) server_negotiation.cc:407] Connection header received
0619 18:49:48.300130 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.300132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.300169 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.300233 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:48.300902 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.301413 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.302418 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.302661 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.302744 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.304067 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:48.304071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:48.304073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:48.304097 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:48.305340 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.305777 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.305780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.305781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.305851 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.306117 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.306119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.306120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.306371 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:48.306473 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.306764 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.306886 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:49:48.310286 26990 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:49:48.313339 26990 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:49:48.327875 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.323046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.12.234.195:37405)
0619 18:49:48.323502 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.323516 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.323541 (+ 25us) server_negotiation.cc:407] Connection header received
0619 18:49:48.323616 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.323622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.323858 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.324016 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:48.324338 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.325131 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.326517 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.327178 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.327297 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.327456 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.327612 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.327713 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":126,"threads_started":1}
I20260619 18:49:48.327868 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.322372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37405 (local address 127.0.0.1:47676)
0619 18:49:48.322985 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.323050 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.323210 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.324027 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.324034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.324055 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:48.324212 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.324219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.325254 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.325258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.326385 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.326394 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.327297 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.327301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.327415 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.327452 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.327565 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.327709 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"spinlock_wait_cycles":14848,"thread_start_us":103,"threads_started":1}
I20260619 18:49:48.329093 27460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.322148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:32801 (local address 127.0.0.1:54488)
0619 18:49:48.323482 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.323495 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.323574 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.324190 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.324193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.324219 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:48.324381 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.324389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.325868 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.325871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.327286 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.327297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.328681 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.328685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.328785 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.328810 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.328887 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.328964 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1281,"mutex_wait_us":603,"spinlock_wait_cycles":4864,"thread_start_us":83,"threads_started":1}
I20260619 18:49:48.329298 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.322414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54488 (local address 127.12.234.193:32801)
0619 18:49:48.323505 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.323521 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.323546 (+ 25us) server_negotiation.cc:407] Connection header received
0619 18:49:48.323730 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.323735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.323879 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.324016 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:48.324499 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.325707 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.327433 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.328557 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.328678 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.328805 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.329044 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.329151 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":969,"thread_start_us":183,"threads_started":1}
I20260619 18:49:48.329790 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.322148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:42583 (local address 127.0.0.1:43312)
0619 18:49:48.322985 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.323050 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.323210 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.324193 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.324196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.324214 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:48.324404 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.324410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.325868 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.325871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.327285 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.327297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.328637 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.328641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.328739 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.328761 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.328825 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.329664 (+ 839us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":188,"spinlock_wait_cycles":6656,"thread_start_us":81,"threads_started":1}
I20260619 18:49:48.330231 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.322348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.12.234.194:42583)
0619 18:49:48.323748 (+ 1400us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.323758 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.323786 (+ 28us) server_negotiation.cc:407] Connection header received
0619 18:49:48.323848 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.323852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.324021 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.324162 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:48.324532 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.325709 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.327434 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.328308 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.328431 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.328571 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.329950 (+ 1379us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.330061 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1295,"thread_start_us":112,"threads_started":1}
I20260619 18:49:48.331485 27383 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 (DEFAULT_TABLE table=test-table [id=babc93c508274d92a2701fd057ca86d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:49:48.332758 27383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:48.332545 27115 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 (DEFAULT_TABLE table=test-table [id=babc93c508274d92a2701fd057ca86d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:49:48.332683 27249 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 (DEFAULT_TABLE table=test-table [id=babc93c508274d92a2701fd057ca86d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:49:48.333580 27115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:48.333604 27249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:48.340584 27466 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Bootstrap starting.
I20260619 18:49:48.341236 27467 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Bootstrap starting.
I20260619 18:49:48.342516 27468 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: Bootstrap starting.
I20260619 18:49:48.343518 27467 tablet_bootstrap.cc:654] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:48.343567 27466 tablet_bootstrap.cc:654] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:48.344452 27466 log.cc:824] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:48.344452 27467 log.cc:824] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:48.344762 27468 tablet_bootstrap.cc:654] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:48.345597 27468 log.cc:824] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:48.346999 27467 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: No bootstrap required, opened a new log
I20260619 18:49:48.347034 27466 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: No bootstrap required, opened a new log
I20260619 18:49:48.347208 27467 ts_tablet_manager.cc:1397] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 18:49:48.347220 27466 ts_tablet_manager.cc:1397] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 18:49:48.347585 27468 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: No bootstrap required, opened a new log
I20260619 18:49:48.347764 27468 ts_tablet_manager.cc:1397] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 18:49:48.350773 27467 raft_consensus.cc:348] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.351013 27467 raft_consensus.cc:374] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:48.351063 27467 raft_consensus.cc:729] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5649f03b3e485bb59f4ddccff6834a, State: Initialized, Role: FOLLOWER
I20260619 18:49:48.351130 27466 raft_consensus.cc:348] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.351408 27466 raft_consensus.cc:374] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:48.351459 27466 raft_consensus.cc:729] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d144ccf030c444f95a120f25d3123ae, State: Initialized, Role: FOLLOWER
I20260619 18:49:48.351358 27468 raft_consensus.cc:348] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.351605 27468 raft_consensus.cc:374] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:48.351521 27467 consensus_queue.cc:260] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.351658 27468 raft_consensus.cc:729] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e037a2bcf6a4b93bc4a5da228b98d30, State: Initialized, Role: FOLLOWER
I20260619 18:49:48.352020 27468 consensus_queue.cc:260] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.352085 27466 consensus_queue.cc:260] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.352569 27467 ts_tablet_manager.cc:1428] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 18:49:48.352933 27181 heartbeater.cc:503] Master 127.12.234.254:46765 was elected leader, sending a full tablet report...
I20260619 18:49:48.352957 27468 ts_tablet_manager.cc:1428] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 18:49:48.352957 27466 ts_tablet_manager.cc:1428] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:49:48.353302 27449 heartbeater.cc:503] Master 127.12.234.254:46765 was elected leader, sending a full tablet report...
I20260619 18:49:48.354058 27315 heartbeater.cc:503] Master 127.12.234.254:46765 was elected leader, sending a full tablet report...
I20260619 18:49:48.357414 27474 raft_consensus.cc:482] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:49:48.357563 27474 raft_consensus.cc:504] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.358615 27474 leader_election.cc:288] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801), 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405)
I20260619 18:49:48.358747 27473 raft_consensus.cc:482] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:49:48.358891 27473 raft_consensus.cc:504] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.361593 27473 leader_election.cc:288] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583), 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801)
I20260619 18:49:48.363327 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.358790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:32801 (local address 127.12.234.194:33487)
0619 18:49:48.359042 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.359056 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.359137 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.359707 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.359710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.359727 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:48.359866 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.359873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.361022 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.361025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.362035 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.362044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.362902 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.362906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.363009 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.363040 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.363115 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.363191 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":24}
I20260619 18:49:48.363561 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.358910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:38939 (local address 127.12.234.195:37405)
0619 18:49:48.359018 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.359022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.359541 (+ 519us) server_negotiation.cc:407] Connection header received
0619 18:49:48.359635 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.359638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.359679 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.359731 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:48.360011 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.360681 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.362451 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.363018 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.363127 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.363170 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.363378 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.363449 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 18:49:48.363663 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.359277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:33487 (local address 127.12.234.193:32801)
0619 18:49:48.359406 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.359411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.359425 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:49:48.359468 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.359471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.359514 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.359583 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:48.359987 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.360689 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.362155 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.362695 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.362799 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.362849 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.363478 (+ 629us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.363553 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 18:49:48.364044 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.358846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37405 (local address 127.12.234.194:38939)
0619 18:49:48.359455 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.359467 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.359552 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.359746 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.359749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.359765 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:48.359886 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.359891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.361178 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.361181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.362343 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.362351 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.363132 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.363136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.363236 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.363262 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.363826 (+ 564us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.363896 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
I20260619 18:49:48.364187 27135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8a4c6887b44b53b9d5ab11dd6a4ce2" candidate_uuid: "3d144ccf030c444f95a120f25d3123ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a5649f03b3e485bb59f4ddccff6834a" is_pre_election: true
I20260619 18:49:48.364506 27135 raft_consensus.cc:2454] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d144ccf030c444f95a120f25d3123ae in term 0.
I20260619 18:49:48.364543 27403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8a4c6887b44b53b9d5ab11dd6a4ce2" candidate_uuid: "3d144ccf030c444f95a120f25d3123ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" is_pre_election: true
I20260619 18:49:48.364725 27403 raft_consensus.cc:2454] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d144ccf030c444f95a120f25d3123ae in term 0.
I20260619 18:49:48.364977 27205 leader_election.cc:302] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5649f03b3e485bb59f4ddccff6834a, 3d144ccf030c444f95a120f25d3123ae; no voters:
I20260619 18:49:48.365272 27474 raft_consensus.cc:2781] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:49:48.365348 27474 raft_consensus.cc:482] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:49:48.365394 27474 raft_consensus.cc:3037] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:48.366014 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.360137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:42583 (local address 127.12.234.195:46991)
0619 18:49:48.360488 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.360501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.360575 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.361083 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.361086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.361101 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:48.361207 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.361214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.362167 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.362171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.364896 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.364901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.365705 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.365707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.365777 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.365791 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.365840 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.365887 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":671}
I20260619 18:49:48.366084 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.360806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:46991 (local address 127.12.234.194:42583)
0619 18:49:48.360902 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.360906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.360916 (+ 10us) server_negotiation.cc:407] Connection header received
0619 18:49:48.360951 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.360954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.361001 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.361067 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:48.361315 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.362039 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.364993 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.365524 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.365591 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.365632 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.365884 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.365983 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 18:49:48.366644 27474 raft_consensus.cc:504] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.366724 27269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8a4c6887b44b53b9d5ab11dd6a4ce2" candidate_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d144ccf030c444f95a120f25d3123ae" is_pre_election: true
I20260619 18:49:48.366951 27269 raft_consensus.cc:2379] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e037a2bcf6a4b93bc4a5da228b98d30 in current term 1: Already voted for candidate 3d144ccf030c444f95a120f25d3123ae in this term.
I20260619 18:49:48.367033 27474 leader_election.cc:288] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 election: Requested vote from peers 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801), 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405)
I20260619 18:49:48.367251 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.360213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:32801 (local address 127.12.234.195:60453)
0619 18:49:48.360838 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.360851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.360920 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.362678 (+ 1758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.362681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.362698 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:48.362786 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.362791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.363738 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.363741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.364776 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.364786 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.366865 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.366869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.366968 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.366992 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.367065 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.367134 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":18,"thread_start_us":95,"threads_started":1}
I20260619 18:49:48.367331 27135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8a4c6887b44b53b9d5ab11dd6a4ce2" candidate_uuid: "3d144ccf030c444f95a120f25d3123ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a5649f03b3e485bb59f4ddccff6834a"
I20260619 18:49:48.367450 27135 raft_consensus.cc:3037] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:48.367540 27403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8a4c6887b44b53b9d5ab11dd6a4ce2" candidate_uuid: "3d144ccf030c444f95a120f25d3123ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30"
I20260619 18:49:48.367652 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.360543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:60453 (local address 127.12.234.193:32801)
0619 18:49:48.360814 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.360824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.360890 (+ 66us) server_negotiation.cc:407] Connection header received
0619 18:49:48.361001 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.361004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.362573 (+ 1569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.362649 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:48.362879 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.363623 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.366152 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.366721 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.366839 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.366883 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.367431 (+ 548us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.367520 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":81,"threads_started":1}
I20260619 18:49:48.367658 27403 raft_consensus.cc:3037] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:48.368502 27135 raft_consensus.cc:2454] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d144ccf030c444f95a120f25d3123ae in term 1.
I20260619 18:49:48.368749 27403 raft_consensus.cc:2454] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d144ccf030c444f95a120f25d3123ae in term 1.
I20260619 18:49:48.368911 27135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8a4c6887b44b53b9d5ab11dd6a4ce2" candidate_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a5649f03b3e485bb59f4ddccff6834a" is_pre_election: true
I20260619 18:49:48.368930 27205 leader_election.cc:302] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5649f03b3e485bb59f4ddccff6834a, 3d144ccf030c444f95a120f25d3123ae; no voters:
I20260619 18:49:48.369076 27135 raft_consensus.cc:2379] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e037a2bcf6a4b93bc4a5da228b98d30 in current term 1: Already voted for candidate 3d144ccf030c444f95a120f25d3123ae in this term.
I20260619 18:49:48.369076 27474 raft_consensus.cc:2781] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:49:48.369256 27474 raft_consensus.cc:686] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 LEADER]: Becoming Leader. State: Replica: 3d144ccf030c444f95a120f25d3123ae, State: Running, Role: LEADER
I20260619 18:49:48.369554 27474 consensus_queue.cc:237] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.369709 27339 leader_election.cc:302] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e037a2bcf6a4b93bc4a5da228b98d30; no voters: 2a5649f03b3e485bb59f4ddccff6834a, 3d144ccf030c444f95a120f25d3123ae
I20260619 18:49:48.369948 27473 raft_consensus.cc:2726] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 18:49:48.372668 26989 catalog_manager.cc:5495] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d144ccf030c444f95a120f25d3123ae (127.12.234.194). New cstate: current_term: 1 leader_uuid: "3d144ccf030c444f95a120f25d3123ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260619 18:49:48.392796 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.389397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.12.234.194:42583)
0619 18:49:48.389534 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.389537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.389550 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:49:48.389633 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.389636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.389690 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.389776 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:49:48.389963 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.390519 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.391322 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.391560 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.391641 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.391962 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:49:48.392403 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:49:48.392503 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.392600 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.392652 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 18:49:48.398818 27135 raft_consensus.cc:1261] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Refusing update from remote peer 3d144ccf030c444f95a120f25d3123ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:49:48.398859 27403 raft_consensus.cc:1261] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 1 FOLLOWER]: Refusing update from remote peer 3d144ccf030c444f95a120f25d3123ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:49:48.399585 27478 consensus_queue.cc:1035] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:49:48.399860 27474 consensus_queue.cc:1035] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:49:48.408538 27482 mvcc.cc:205] Tried to move back new op lower bound from 7298641872474071040 to 7298641872364359680. Current Snapshot: MvccSnapshot[applied={T|T < 7298641872474071040}]
I20260619 18:49:48.408434 27481 mvcc.cc:205] Tried to move back new op lower bound from 7298641872474071040 to 7298641872364359680. Current Snapshot: MvccSnapshot[applied={T|T < 7298641872474071040}]
I20260619 18:49:48.420753 26988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:49:48.420969 26988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:49:48.423704 26988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Sending DeleteTablet for 3 replicas of tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2
I20260619 18:49:48.424603 27249 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:48 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:43312
I20260619 18:49:48.424638 27115 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:48 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:54488
I20260619 18:49:48.424907 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 27052
I20260619 18:49:48.425088 27383 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:48 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:47676
I20260619 18:49:48.425637 27492 tablet_replica.cc:330] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: stopping tablet replica
I20260619 18:49:48.425706 27491 tablet_replica.cc:330] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: stopping tablet replica
I20260619 18:49:48.425999 27492 raft_consensus.cc:2229] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:49:48.426141 27491 raft_consensus.cc:2229] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:49:48.426447 27492 raft_consensus.cc:2258] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:49:48.426606 27491 raft_consensus.cc:2258] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:49:48.428532 27492 ts_tablet_manager.cc:1905] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:49:48.431469 27492 ts_tablet_manager.cc:1918] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:49:48.431592 27492 log.cc:1196] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/6e8a4c6887b44b53b9d5ab11dd6a4ce2
I20260619 18:49:48.431928 27491 ts_tablet_manager.cc:1905] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:49:48.431977 27492 ts_tablet_manager.cc:1939] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 4e037a2bcf6a4b93bc4a5da228b98d30: Deleting consensus metadata
W20260619 18:49:48.432685 26977 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:32801 (error 108)
I20260619 18:49:48.433355 26975 catalog_manager.cc:4841] TS 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405): tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 (table test-table [id=babc93c508274d92a2701fd057ca86d3]) successfully deleted
I20260619 18:49:48.433727 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 27186
I20260619 18:49:48.435555 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.435244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:32801 (local address 127.0.0.1:54504)
0619 18:49:48.435404 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.435476 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:32801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260619 18:49:48.435721 26977 catalog_manager.cc:4584] TS 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2: Network error: Client connection negotiation failed: client connection to 127.12.234.193:32801: connect: Connection refused (error 111)
I20260619 18:49:48.439054 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 27320
I20260619 18:49:48.440922 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.440631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:42583 (local address 127.0.0.1:43328)
0619 18:49:48.440772 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.440835 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.194:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 18:49:48.443960 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 26958
I20260619 18:49:48.449721 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46765
--webserver_interface=127.12.234.254
--webserver_port=37727
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:46765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:48.565986 27494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:48.566358 27494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:48.566419 27494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:48.570708 27494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:49:48.570786 27494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:48.570806 27494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:48.570822 27494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:49:48.570837 27494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:49:48.576562 27494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:46765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=37727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:48.577796 27494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:48.578914 27494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:49:48.593959 27494 server_base.cc:1034] running on GCE node
W20260619 18:49:48.593888 27503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:48.593876 27500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:48.593886 27501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:48.594853 27494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:48.596118 27494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:48.597301 27494 hybrid_clock.cc:648] HybridClock initialized: now 1781894988597241 us; error 79 us; skew 500 ppm
Jun 19 18:49:48 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894988, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:48.600307 27494 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:49:48.601316 27494 webserver.cc:460] Webserver started at http://127.12.234.254:37727/ using document root <none> and password file <none>
I20260619 18:49:48.601912 27494 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:48.601995 27494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:48.606310 27494 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 18:49:48.608670 27510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:48.610035 27494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:49:48.610210 27494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "ca10f8244b5e4b6a9d775ee293671288"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "91d25e0688cd48c9dc1d0e6b8c2b1a4c"
server_key_iv: "1039074227a0a21ac206e0f2b1afadff"
server_key_version: "encryptionkey@0"
I20260619 18:49:48.610891 27494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:48.625072 27494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:48.628854 27494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:48.629065 27494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:48.636266 27494 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:46765
I20260619 18:49:48.636548 27562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:46765 every 8 connection(s)
I20260619 18:49:48.637517 27494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:49:48.642102 27563 sys_catalog.cc:263] Verifying existing consensus state
I20260619 18:49:48.644002 27563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Bootstrap starting.
I20260619 18:49:48.646488 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 27494
I20260619 18:49:48.646950 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:32801
--local_ip_for_outbound_sockets=127.12.234.193
--tserver_master_addrs=127.12.234.254:46765
--webserver_port=45003
--webserver_interface=127.12.234.193
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:49:48.662418 27563 log.cc:824] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:48.672040 27563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:49:48.672407 27563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Bootstrap complete.
I20260619 18:49:48.676849 27563 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:48.677287 27563 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca10f8244b5e4b6a9d775ee293671288, State: Initialized, Role: FOLLOWER
I20260619 18:49:48.677775 27563 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:48.677897 27563 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:49:48.677981 27563 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:49:48.678071 27563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 1 FOLLOWER]: Advancing to term 2
I20260619 18:49:48.680100 27563 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:48.680439 27563 leader_election.cc:302] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca10f8244b5e4b6a9d775ee293671288; no voters:
I20260619 18:49:48.680742 27563 leader_election.cc:288] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 18:49:48.680857 27567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 18:49:48.681053 27567 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 2 LEADER]: Becoming Leader. State: Replica: ca10f8244b5e4b6a9d775ee293671288, State: Running, Role: LEADER
I20260619 18:49:48.681458 27567 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:48.682091 27563 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:49:48.683007 27568 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ca10f8244b5e4b6a9d775ee293671288" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } } }
I20260619 18:49:48.683157 27568 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:48.683452 27569 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca10f8244b5e4b6a9d775ee293671288. Latest consensus state: current_term: 2 leader_uuid: "ca10f8244b5e4b6a9d775ee293671288" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } } }
I20260619 18:49:48.683547 27569 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:48.683636 27576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:49:48.688959 27576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:49:48.689999 27576 catalog_manager.cc:1259] Loaded cluster ID: 63648c854de848c49a3ef21cc9c796b6
I20260619 18:49:48.690073 27576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:49:48.691574 27576 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 18:49:48.696377 27576 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 18:49:48.702502 27576 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:49:48.703372 27576 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Loaded TSK: 0
I20260619 18:49:48.704659 27576 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 18:49:48.767524 27565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:48.767819 27565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:48.767882 27565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:48.772104 27565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:48.772178 27565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:48.772259 27565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:49:48.777789 27565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:32801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=45003
--tserver_master_addrs=127.12.234.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:48.778936 27565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:48.779950 27565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:48.790119 27591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:48.790129 27590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:48.790277 27593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:48.790583 27565 server_base.cc:1034] running on GCE node
I20260619 18:49:48.791093 27565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:48.791961 27565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:48.793213 27565 hybrid_clock.cc:648] HybridClock initialized: now 1781894988793139 us; error 55 us; skew 500 ppm
Jun 19 18:49:48 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894988, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:48.797068 27565 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:49:48.798529 27565 webserver.cc:460] Webserver started at http://127.12.234.193:45003/ using document root <none> and password file <none>
I20260619 18:49:48.799340 27565 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:48.799463 27565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:48.805109 27565 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:49:48.808007 27600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:48.809643 27565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:49:48.809787 27565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "2a5649f03b3e485bb59f4ddccff6834a"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "381461cfdcc55f7101baad2232611fbe"
server_key_iv: "f0b0457ce212862b596f91af252aa48f"
server_key_version: "encryptionkey@0"
I20260619 18:49:48.810601 27565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:48.826485 27565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:48.830641 27565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:48.830868 27565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:48.831558 27565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:48.832852 27607 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 18:49:48.837124 27565 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 18:49:48.837211 27565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260619 18:49:48.837271 27565 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 18:49:48.839378 27565 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 18:49:48.839442 27565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260619 18:49:48.839809 27607 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Bootstrap starting.
I20260619 18:49:48.849359 27565 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:32801
I20260619 18:49:48.849439 27714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:32801 every 8 connection(s)
I20260619 18:49:48.850421 27565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:49:48.854427 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 27565
I20260619 18:49:48.854918 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:42583
--local_ip_for_outbound_sockets=127.12.234.194
--tserver_master_addrs=127.12.234.254:46765
--webserver_port=36313
--webserver_interface=127.12.234.194
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:49:48.858150 27607 log.cc:824] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Log is configured to *not* fsync() on all Append() calls
Jun 19 18:49:48 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894988, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:48.866405 27607 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:49:48.866832 27607 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Bootstrap complete.
I20260619 18:49:48.867959 27607 ts_tablet_manager.cc:1397] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Time spent bootstrapping tablet: real 0.028s user 0.018s sys 0.008s
I20260619 18:49:48.870298 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.852453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:43385)
0619 18:49:48.853106 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.853175 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.854372 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.856000 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.856010 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.856540 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:48.857209 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.857232 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.858698 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.858702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.860027 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.860040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.861057 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.861062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.861283 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.861979 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:48.862003 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:48.864096 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:48.866722 (+ 2626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:48.866730 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:48.866745 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:48.867104 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:48.867510 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:48.867513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:48.867515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:48.867692 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:48.868394 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:48.868401 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:48.868721 (+ 320us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.868922 (+ 201us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.869280 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":124,"threads_started":1}
I20260619 18:49:48.870379 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.852456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:43385 (local address 127.12.234.254:46765)
0619 18:49:48.852853 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.852872 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.853326 (+ 454us) server_negotiation.cc:407] Connection header received
0619 18:49:48.854960 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.854978 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.855454 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.855849 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:48.857416 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.858518 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.860167 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.860884 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.861048 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.864351 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:48.864358 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:48.864375 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:48.864412 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:48.866566 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.867248 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.867254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.867260 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.867418 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.867777 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.867779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.867781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.868278 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:48.868409 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.869071 (+ 662us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.869394 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":122,"threads_started":1}
I20260619 18:49:48.870936 27607 raft_consensus.cc:348] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.871439 27607 raft_consensus.cc:729] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5649f03b3e485bb59f4ddccff6834a, State: Initialized, Role: FOLLOWER
I20260619 18:49:48.871908 27607 consensus_queue.cc:260] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:48.872937 27607 ts_tablet_manager.cc:1428] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 18:49:48.873317 27715 heartbeater.cc:347] Connected to a master server at 127.12.234.254:46765
I20260619 18:49:48.873526 27715 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:48.874014 27715 heartbeater.cc:511] Master 127.12.234.254:46765 requested a full tablet report, sending...
I20260619 18:49:48.876416 27527 ts_manager.cc:194] Registered new tserver with Master: 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801)
I20260619 18:49:48.879767 27527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:43385
I20260619 18:49:48.882505 27715 heartbeater.cc:503] Master 127.12.234.254:46765 was elected leader, sending a full tablet report...
Jun 19 18:49:48 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894988, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:49:48.890123 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.879301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:32801 (local address 127.0.0.1:60148)
0619 18:49:48.880172 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:48.880219 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:48.880357 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:48.881029 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:48.881035 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:48.881460 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:48.881591 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.881611 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.882980 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.882984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.883778 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:48.883787 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.884492 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.884495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:48.884609 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.885156 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:48.885167 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:48.886717 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:48.888365 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:48.888373 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:48.888379 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:48.888599 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:48.888872 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:48.888875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:48.888877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:48.888958 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:48.889431 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:48.889437 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:48.889559 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 18:49:48.889678 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 18:49:48.889936 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":114,"threads_started":1}
I20260619 18:49:48.890118 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:48.879407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60148 (local address 127.12.234.193:32801)
0619 18:49:48.879827 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:48.879839 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:48.880353 (+ 514us) server_negotiation.cc:407] Connection header received
0619 18:49:48.880506 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:48.880511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:48.880738 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:48.880868 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:48.881737 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.882858 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.883894 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:48.884378 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:48.884471 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:48.886854 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:48.886865 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:48.886877 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:48.886899 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:48.888263 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.888703 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.888709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.888711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.888791 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:48.889033 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:48.889036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:48.889038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:48.889329 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:48.889457 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:48.889769 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 18:49:48.889933 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":109,"threads_started":1}
I20260619 18:49:48.891187 27649 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:48 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:60148
I20260619 18:49:48.892050 27724 tablet_replica.cc:330] stopping tablet replica
I20260619 18:49:48.892385 27724 raft_consensus.cc:2229] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:49:48.892660 27724 raft_consensus.cc:2258] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:49:48.893778 27724 ts_tablet_manager.cc:1905] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:49:48.898533 27724 ts_tablet_manager.cc:1918] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:49:48.898662 27724 log.cc:1196] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/6e8a4c6887b44b53b9d5ab11dd6a4ce2
I20260619 18:49:48.898918 27724 ts_tablet_manager.cc:1939] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 2a5649f03b3e485bb59f4ddccff6834a: Deleting consensus metadata
I20260619 18:49:48.901011 27514 catalog_manager.cc:4841] TS 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801): tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 (table test-table [id=babc93c508274d92a2701fd057ca86d3]) successfully deleted
W20260619 18:49:48.976526 27719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:48.976809 27719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:48.976853 27719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:48.980976 27719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:48.981065 27719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:48.981128 27719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:49:48.987067 27719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:42583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=36313
--tserver_master_addrs=127.12.234.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:48.988174 27719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:48.989243 27719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:48.999610 27729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:48.999655 27730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:48.999603 27732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:48.999953 27719 server_base.cc:1034] running on GCE node
I20260619 18:49:49.000473 27719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:49.001179 27719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:49.002331 27719 hybrid_clock.cc:648] HybridClock initialized: now 1781894989002306 us; error 40 us; skew 500 ppm
Jun 19 18:49:49 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894989, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:49.005218 27719 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:49:49.006142 27719 webserver.cc:460] Webserver started at http://127.12.234.194:36313/ using document root <none> and password file <none>
I20260619 18:49:49.006769 27719 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:49.006858 27719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:49.010941 27719 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260619 18:49:49.013295 27739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:49.014710 27719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 18:49:49.014823 27719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "3d144ccf030c444f95a120f25d3123ae"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "fc40cf871a8a6352f74ba1f56d088090"
server_key_iv: "588b7cddff268a534c08820d07598c2b"
server_key_version: "encryptionkey@0"
I20260619 18:49:49.015372 27719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:49.029228 27719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:49.032594 27719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:49.032801 27719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:49.033473 27719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:49.034655 27746 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 18:49:49.038946 27719 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 18:49:49.039032 27719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260619 18:49:49.039105 27719 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 18:49:49.040885 27719 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 18:49:49.040937 27719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 18:49:49.041229 27746 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Bootstrap starting.
I20260619 18:49:49.047221 27746 log.cc:824] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:49.050411 27719 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:42583
I20260619 18:49:49.050511 27854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:42583 every 8 connection(s)
I20260619 18:49:49.051306 27719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:49:49.053231 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 27719
I20260619 18:49:49.053704 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:37405
--local_ip_for_outbound_sockets=127.12.234.195
--tserver_master_addrs=127.12.234.254:46765
--webserver_port=33409
--webserver_interface=127.12.234.195
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:49:49.056191 27746 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:49:49.056610 27746 tablet_bootstrap.cc:492] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Bootstrap complete.
I20260619 18:49:49.056972 27746 ts_tablet_manager.cc:1397] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
Jun 19 18:49:49 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894989, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:49.060777 27746 raft_consensus.cc:348] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:49.061354 27746 raft_consensus.cc:729] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d144ccf030c444f95a120f25d3123ae, State: Initialized, Role: FOLLOWER
I20260619 18:49:49.061923 27746 consensus_queue.cc:260] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } }
I20260619 18:49:49.063092 27746 ts_tablet_manager.cc:1428] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260619 18:49:49.065088 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.053185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:56755 (local address 127.12.234.254:46765)
0619 18:49:49.053343 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:49.053348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:49.054227 (+ 879us) server_negotiation.cc:407] Connection header received
0619 18:49:49.055359 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:49.055363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:49.055427 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:49.055533 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:49.056998 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.057640 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.058907 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.059155 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.059245 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.061656 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:49.061660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:49.061663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:49.061694 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:49.063160 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:49.063650 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:49.063653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:49.063654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:49.063730 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:49.064034 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:49.064038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:49.064039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:49.064224 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:49.064308 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:49.064781 (+ 473us) server_negotiation.cc:299] Negotiation successful
0619 18:49:49.064923 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 18:49:49.065864 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.053343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:56755)
0619 18:49:49.054042 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.054096 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:49.055061 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:49.055722 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:49.055731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:49.056215 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:49.056833 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.056850 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.057784 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.057787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.058780 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.058790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.059277 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.059281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.059456 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.060022 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:49.060037 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:49.061463 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:49.063259 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:49.063264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:49.063280 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:49.063530 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:49.063810 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:49.063814 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:49.063816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:49.063951 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:49.064317 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:49.064324 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:49.064561 (+ 237us) client_negotiation.cc:769] Sending connection context
0619 18:49:49.064726 (+ 165us) client_negotiation.cc:240] Negotiation successful
0619 18:49:49.065043 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":135,"threads_started":1}
I20260619 18:49:49.067121 27855 heartbeater.cc:347] Connected to a master server at 127.12.234.254:46765
I20260619 18:49:49.067339 27855 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:49.067853 27855 heartbeater.cc:511] Master 127.12.234.254:46765 requested a full tablet report, sending...
I20260619 18:49:49.069517 27527 ts_manager.cc:194] Registered new tserver with Master: 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583)
I20260619 18:49:49.070623 27527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:56755
I20260619 18:49:49.073101 27855 heartbeater.cc:503] Master 127.12.234.254:46765 was elected leader, sending a full tablet report...
Jun 19 18:49:49 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894988, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.194@KRBTEST.COM
I20260619 18:49:49.080533 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.070437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:42583 (local address 127.0.0.1:53058)
0619 18:49:49.070614 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.070627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:49.070700 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:49.071544 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:49.071547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:49.071742 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:49.071853 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.071860 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.073165 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.073168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.074024 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.074035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.074861 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.074866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.074975 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.075500 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:49.075514 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:49.076946 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:49.078946 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:49.078950 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:49.078952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:49.079218 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:49.079487 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:49.079490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:49.079492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:49.079581 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:49.079983 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:49.079985 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:49.080092 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 18:49:49.080206 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 18:49:49.080361 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260619 18:49:49.080574 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.070846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.12.234.194:42583)
0619 18:49:49.071124 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:49.071133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:49.071151 (+ 18us) server_negotiation.cc:407] Connection header received
0619 18:49:49.071205 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:49.071209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:49.071410 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:49.071543 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:49.071966 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.073034 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.074164 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.074720 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.074800 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.077127 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:49.077138 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:49.077151 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:49.077174 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:49.078826 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:49.079324 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:49.079329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:49.079330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:49.079410 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:49.079656 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:49.079658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:49.079659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:49.079898 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:49.080004 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:49.080287 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 18:49:49.080433 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":197,"thread_start_us":110,"threads_started":1}
I20260619 18:49:49.081717 27788 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:48 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:53058
I20260619 18:49:49.082685 27861 tablet_replica.cc:330] stopping tablet replica
I20260619 18:49:49.082998 27861 raft_consensus.cc:2229] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:49:49.083263 27861 raft_consensus.cc:2258] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:49:49.084246 27861 ts_tablet_manager.cc:1905] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:49:49.088083 27861 ts_tablet_manager.cc:1918] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:49:49.088227 27861 log.cc:1196] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/6e8a4c6887b44b53b9d5ab11dd6a4ce2
I20260619 18:49:49.088474 27861 ts_tablet_manager.cc:1939] T 6e8a4c6887b44b53b9d5ab11dd6a4ce2 P 3d144ccf030c444f95a120f25d3123ae: Deleting consensus metadata
I20260619 18:49:49.090118 27513 catalog_manager.cc:4841] TS 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583): tablet 6e8a4c6887b44b53b9d5ab11dd6a4ce2 (table test-table [id=babc93c508274d92a2701fd057ca86d3]) successfully deleted
W20260619 18:49:49.173336 27858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:49.173640 27858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:49.173710 27858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:49.178006 27858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:49.178094 27858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:49.178164 27858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:49:49.184082 27858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:37405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=33409
--tserver_master_addrs=127.12.234.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:49.185252 27858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:49.186349 27858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:49.196532 27866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:49.196767 27858 server_base.cc:1034] running on GCE node
W20260619 18:49:49.196517 27869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:49.196518 27867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:49.197331 27858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:49.198082 27858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:49.199254 27858 hybrid_clock.cc:648] HybridClock initialized: now 1781894989199219 us; error 47 us; skew 500 ppm
Jun 19 18:49:49 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894989, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:49.202096 27858 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:49:49.203078 27858 webserver.cc:460] Webserver started at http://127.12.234.195:33409/ using document root <none> and password file <none>
I20260619 18:49:49.203644 27858 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:49.203765 27858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:49.207962 27858 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:49:49.210251 27876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:49.211591 27858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:49:49.211746 27858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "4e037a2bcf6a4b93bc4a5da228b98d30"
format_stamp: "Formatted at 2026-06-19 18:49:48 on dist-test-slave-fljv"
server_key: "2b2c61428031eb334206ce9cf2377daa"
server_key_iv: "d00587a7715682580f0299e3736ba852"
server_key_version: "encryptionkey@0"
I20260619 18:49:49.212324 27858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:49.237044 27858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:49.240422 27858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:49.240612 27858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:49.241284 27858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:49.242158 27858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:49.242233 27858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:49.242290 27858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:49.242318 27858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:49.250900 27858 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:37405
I20260619 18:49:49.250931 27989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:37405 every 8 connection(s)
I20260619 18:49:49.251937 27858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:49:49.261399 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 27858
Jun 19 18:49:49 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894989, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:49:49.265833 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.253820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:37857 (local address 127.12.234.254:46765)
0619 18:49:49.253970 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:49.253975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:49.254957 (+ 982us) server_negotiation.cc:407] Connection header received
0619 18:49:49.256294 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:49.256297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:49.256351 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:49.256442 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:49.257625 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.258196 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.259314 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.259605 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.259695 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.262275 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:49.262279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:49.262281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:49.262308 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:49.263642 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:49.264244 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:49.264247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:49.264248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:49.264324 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:49.264706 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:49.264708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:49.264710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:49.264888 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:49.264975 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:49.265527 (+ 552us) server_negotiation.cc:299] Negotiation successful
0619 18:49:49.265668 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 18:49:49.266805 27992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.254312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:37857)
0619 18:49:49.254791 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.254834 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:49.256060 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:49.256596 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:49.256603 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:49.256963 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:49.257472 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.257485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.258302 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.258304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.259196 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.259204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.259738 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.259741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.259914 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.260487 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:49.260502 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:49.262076 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:49.263768 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:49.263776 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:49.263794 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:49.264153 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:49.264415 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:49.264418 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:49.264421 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:49.264577 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:49.264993 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:49.265001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:49.265283 (+ 282us) client_negotiation.cc:769] Sending connection context
0619 18:49:49.265466 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 18:49:49.265832 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":127,"threads_started":1}
I20260619 18:49:49.268170 27990 heartbeater.cc:347] Connected to a master server at 127.12.234.254:46765
I20260619 18:49:49.268435 27990 heartbeater.cc:464] Registering TS with master...
I20260619 18:49:49.269063 27990 heartbeater.cc:511] Master 127.12.234.254:46765 requested a full tablet report, sending...
I20260619 18:49:49.270009 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.262074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47564 (local address 127.12.234.254:46765)
0619 18:49:49.262350 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:49.262355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:49.262570 (+ 215us) server_negotiation.cc:407] Connection header received
0619 18:49:49.262683 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:49.262686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:49.262729 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:49.262799 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:49.263497 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.264253 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.265091 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.265321 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.265404 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.266858 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:49.266862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:49.266865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:49.266892 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:49.268367 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:49.268823 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:49.268826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:49.268827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:49.268901 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:49.269182 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:49.269185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:49.269187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:49.269371 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:49.269519 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:49.269737 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 18:49:49.269869 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":103,"threads_started":1}
I20260619 18:49:49.270279 27527 ts_manager.cc:194] Registered new tserver with Master: 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405)
I20260619 18:49:49.270951 27527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:37857
I20260619 18:49:49.271824 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:49:49.276152 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.272412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47580 (local address 127.12.234.254:46765)
0619 18:49:49.272518 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:49.272522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:49.272712 (+ 190us) server_negotiation.cc:407] Connection header received
0619 18:49:49.272858 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:49.272861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:49.272955 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:49.273042 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:49:49.273279 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.273785 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.274695 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.274997 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.275086 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.275408 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:49:49.275788 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:49:49.275893 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:49.275958 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 18:49:49.276018 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 18:49:49.276789 27524 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:49:49.280117 27524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:49:49.287951 27788 tablet_service.cc:1467] Processing CreateTablet for tablet 8a343af846674f178514fc885509b842 (DEFAULT_TABLE table=test-table [id=76303ed666bb4227a3db8f5398d58ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:49:49.287951 27649 tablet_service.cc:1467] Processing CreateTablet for tablet 8a343af846674f178514fc885509b842 (DEFAULT_TABLE table=test-table [id=76303ed666bb4227a3db8f5398d58ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:49:49.288645 27649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a343af846674f178514fc885509b842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:49.288642 27788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a343af846674f178514fc885509b842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:49.291553 27607 tablet_bootstrap.cc:492] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Bootstrap starting.
I20260619 18:49:49.291947 27746 tablet_bootstrap.cc:492] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: Bootstrap starting.
I20260619 18:49:49.292335 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.287113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37405 (local address 127.0.0.1:48236)
0619 18:49:49.287279 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.287292 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:49.287371 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:49.288319 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:49.288322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:49.288341 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:49.288438 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.288443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.290077 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.290081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.290940 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.290950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.291874 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.291878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.291977 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.292031 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 18:49:49.292115 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 18:49:49.292196 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260619 18:49:49.292392 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.287534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.12.234.195:37405)
0619 18:49:49.287854 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:49.287864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:49.287884 (+ 20us) server_negotiation.cc:407] Connection header received
0619 18:49:49.287950 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:49.287955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:49.288138 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:49.288570 (+ 432us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:49.288753 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.289956 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.291058 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.291760 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.291883 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.292030 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:49.292149 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 18:49:49.292246 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":119,"threads_started":1}
I20260619 18:49:49.292572 27607 tablet_bootstrap.cc:654] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:49.292847 27746 tablet_bootstrap.cc:654] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:49.294154 27607 tablet_bootstrap.cc:492] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: No bootstrap required, opened a new log
I20260619 18:49:49.294301 27607 ts_tablet_manager.cc:1397] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260619 18:49:49.294768 27607 raft_consensus.cc:348] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.294872 27607 raft_consensus.cc:374] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:49.294909 27607 raft_consensus.cc:729] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5649f03b3e485bb59f4ddccff6834a, State: Initialized, Role: FOLLOWER
I20260619 18:49:49.294981 27746 tablet_bootstrap.cc:492] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: No bootstrap required, opened a new log
I20260619 18:49:49.295064 27746 ts_tablet_manager.cc:1397] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 18:49:49.295029 27607 consensus_queue.cc:260] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.295279 27607 ts_tablet_manager.cc:1428] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 18:49:49.295493 27746 raft_consensus.cc:348] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.295608 27746 raft_consensus.cc:374] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:49.295644 27746 raft_consensus.cc:729] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d144ccf030c444f95a120f25d3123ae, State: Initialized, Role: FOLLOWER
I20260619 18:49:49.295758 27746 consensus_queue.cc:260] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.295991 27746 ts_tablet_manager.cc:1428] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 18:49:49.296066 27924 tablet_service.cc:1467] Processing CreateTablet for tablet 8a343af846674f178514fc885509b842 (DEFAULT_TABLE table=test-table [id=76303ed666bb4227a3db8f5398d58ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:49:49.297238 27924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a343af846674f178514fc885509b842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:49:49.300086 27859 raft_consensus.cc:482] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:49:49.300251 27859 raft_consensus.cc:504] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.301347 27859 leader_election.cc:288] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405), 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801)
W20260619 18:49:49.302232 27856 tablet.cc:2369] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:49:49.305526 28003 tablet_bootstrap.cc:492] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: Bootstrap starting.
I20260619 18:49:49.306432 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.301524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37405 (local address 127.12.234.194:38447)
0619 18:49:49.301871 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.301884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:49.302022 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:49.302438 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:49.302440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:49.302456 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:49.302579 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.302585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.303686 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.303689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.304932 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.304945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.305890 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.305894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.305989 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.306041 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 18:49:49.306111 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 18:49:49.306259 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":45,"spinlock_wait_cycles":34816}
I20260619 18:49:49.306605 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.301590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:32801 (local address 127.12.234.194:39437)
0619 18:49:49.301892 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.301906 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:49.302025 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:49.302528 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:49.302531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:49.302548 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:49:49.302681 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.302687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.303666 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.303669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.304951 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:49.304958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.306143 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.306147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:49.306286 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.306304 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 18:49:49.306371 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 18:49:49.306471 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":47,"spinlock_wait_cycles":7936,"thread_start_us":119,"threads_started":1}
I20260619 18:49:49.306600 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.301690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:39437 (local address 127.12.234.193:32801)
0619 18:49:49.301814 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:49.301818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:49.302144 (+ 326us) server_negotiation.cc:407] Connection header received
0619 18:49:49.302223 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:49.302226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:49.302278 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:49.302353 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:49.302784 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.303552 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.305067 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.305731 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.305852 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.305996 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:49.306403 (+ 407us) server_negotiation.cc:299] Negotiation successful
0619 18:49:49.306489 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 18:49:49.306784 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.301779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:38447 (local address 127.12.234.195:37405)
0619 18:49:49.301894 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:49.301899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:49.301960 (+ 61us) server_negotiation.cc:407] Connection header received
0619 18:49:49.302162 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:49.302165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:49.302249 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:49.302300 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:49:49.302719 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.303511 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.305081 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.305641 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.305755 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.305808 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:49.306594 (+ 786us) server_negotiation.cc:299] Negotiation successful
0619 18:49:49.306677 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:49:49.307556 27668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a343af846674f178514fc885509b842" candidate_uuid: "3d144ccf030c444f95a120f25d3123ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a5649f03b3e485bb59f4ddccff6834a" is_pre_election: true
I20260619 18:49:49.307848 27668 raft_consensus.cc:2454] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d144ccf030c444f95a120f25d3123ae in term 0.
I20260619 18:49:49.308285 27743 leader_election.cc:302] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5649f03b3e485bb59f4ddccff6834a, 3d144ccf030c444f95a120f25d3123ae; no voters:
I20260619 18:49:49.308537 28003 tablet_bootstrap.cc:654] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: Neither blocks nor log segments found. Creating new log.
I20260619 18:49:49.308531 27859 raft_consensus.cc:2781] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:49:49.308609 27859 raft_consensus.cc:482] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:49:49.308656 27859 raft_consensus.cc:3037] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:49.307313 27944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a343af846674f178514fc885509b842" candidate_uuid: "3d144ccf030c444f95a120f25d3123ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" is_pre_election: true
I20260619 18:49:49.309454 28003 log.cc:824] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: Log is configured to *not* fsync() on all Append() calls
W20260619 18:49:49.309774 27741 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260619 18:49:49.309828 27859 raft_consensus.cc:504] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.310200 27859 leader_election.cc:288] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 election: Requested vote from peers 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405), 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801)
I20260619 18:49:49.310585 27944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a343af846674f178514fc885509b842" candidate_uuid: "3d144ccf030c444f95a120f25d3123ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30"
I20260619 18:49:49.310840 27668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a343af846674f178514fc885509b842" candidate_uuid: "3d144ccf030c444f95a120f25d3123ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a5649f03b3e485bb59f4ddccff6834a"
W20260619 18:49:49.310962 27741 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260619 18:49:49.311038 27668 raft_consensus.cc:3037] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:49.311996 28003 tablet_bootstrap.cc:492] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: No bootstrap required, opened a new log
I20260619 18:49:49.312189 28003 ts_tablet_manager.cc:1397] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 18:49:49.312183 27668 raft_consensus.cc:2454] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d144ccf030c444f95a120f25d3123ae in term 1.
I20260619 18:49:49.312461 27743 leader_election.cc:302] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a5649f03b3e485bb59f4ddccff6834a, 3d144ccf030c444f95a120f25d3123ae; no voters: 4e037a2bcf6a4b93bc4a5da228b98d30
I20260619 18:49:49.312644 27859 raft_consensus.cc:2781] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:49:49.312898 27859 raft_consensus.cc:686] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 1 LEADER]: Becoming Leader. State: Replica: 3d144ccf030c444f95a120f25d3123ae, State: Running, Role: LEADER
I20260619 18:49:49.313174 27859 consensus_queue.cc:237] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.314952 28003 raft_consensus.cc:348] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.315269 28003 raft_consensus.cc:374] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:49:49.315320 28003 raft_consensus.cc:729] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e037a2bcf6a4b93bc4a5da228b98d30, State: Initialized, Role: FOLLOWER
I20260619 18:49:49.315853 28003 consensus_queue.cc:260] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.316551 27990 heartbeater.cc:503] Master 127.12.234.254:46765 was elected leader, sending a full tablet report...
I20260619 18:49:49.316411 27524 catalog_manager.cc:5495] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d144ccf030c444f95a120f25d3123ae (127.12.234.194). New cstate: current_term: 1 leader_uuid: "3d144ccf030c444f95a120f25d3123ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } health_report { overall_health: HEALTHY } } }
I20260619 18:49:49.317170 28003 ts_tablet_manager.cc:1428] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 18:49:49.331537 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.327821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53066 (local address 127.12.234.194:42583)
0619 18:49:49.327978 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:49.327982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:49.327997 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:49:49.328075 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:49.328077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:49.328133 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:49.328251 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:49:49.328511 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.329031 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.329924 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:49.330164 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:49.330299 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:49.330558 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:49:49.331086 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:49:49.331255 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:49.331327 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 18:49:49.331388 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 18:49:49.335680 27944 raft_consensus.cc:3037] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:49:49.335927 27668 raft_consensus.cc:1261] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Refusing update from remote peer 3d144ccf030c444f95a120f25d3123ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:49:49.336623 27859 consensus_queue.cc:1035] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:49:49.337227 27944 raft_consensus.cc:1261] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 1 FOLLOWER]: Refusing update from remote peer 3d144ccf030c444f95a120f25d3123ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:49:49.337908 27859 consensus_queue.cc:1035] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:49:49.342653 28008 mvcc.cc:205] Tried to move back new op lower bound from 7298641876312109056 to 7298641876229304320. Current Snapshot: MvccSnapshot[applied={T|T < 7298641876312109056}]
I20260619 18:49:49.344834 28009 mvcc.cc:205] Tried to move back new op lower bound from 7298641876312109056 to 7298641876229304320. Current Snapshot: MvccSnapshot[applied={T|T < 7298641876312109056}]
I20260619 18:49:49.353361 27527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:49:49.353551 27527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:49:49.356014 27527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Sending DeleteTablet for 3 replicas of tablet 8a343af846674f178514fc885509b842
I20260619 18:49:49.356813 27649 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a343af846674f178514fc885509b842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:49 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:60148
I20260619 18:49:49.356848 27924 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a343af846674f178514fc885509b842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:49 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:48236
I20260619 18:49:49.357089 27724 tablet_replica.cc:330] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: stopping tablet replica
I20260619 18:49:49.357201 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 27565
I20260619 18:49:49.357231 27724 raft_consensus.cc:2229] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:49:49.357370 27724 raft_consensus.cc:2258] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:49:49.356824 27788 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a343af846674f178514fc885509b842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:49:49 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:53058
I20260619 18:49:49.358035 27861 tablet_replica.cc:330] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: stopping tablet replica
I20260619 18:49:49.358126 27861 raft_consensus.cc:2229] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:49:49.358480 27861 raft_consensus.cc:2258] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:49:49.359423 27861 ts_tablet_manager.cc:1905] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:49:49.362248 27861 ts_tablet_manager.cc:1918] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:49:49.362329 27861 log.cc:1196] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/8a343af846674f178514fc885509b842
I20260619 18:49:49.362566 27861 ts_tablet_manager.cc:1939] T 8a343af846674f178514fc885509b842 P 3d144ccf030c444f95a120f25d3123ae: Deleting consensus metadata
W20260619 18:49:49.362645 27514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:32801 (error 108)
I20260619 18:49:49.362987 28018 tablet_replica.cc:330] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: stopping tablet replica
I20260619 18:49:49.363613 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 27719
I20260619 18:49:49.363615 27513 catalog_manager.cc:4841] TS 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583): tablet 8a343af846674f178514fc885509b842 (table test-table [id=76303ed666bb4227a3db8f5398d58ccd]) successfully deleted
I20260619 18:49:49.364171 28018 raft_consensus.cc:2229] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:49:49.364517 28018 raft_consensus.cc:2258] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:49:49.364938 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.363923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:32801 (local address 127.0.0.1:60158)
0619 18:49:49.364769 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.364851 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:32801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":754}
I20260619 18:49:49.366094 28018 ts_tablet_manager.cc:1905] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 18:49:49.365938 27514 catalog_manager.cc:4584] TS 2a5649f03b3e485bb59f4ddccff6834a (127.12.234.193:32801): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a343af846674f178514fc885509b842: Network error: Client connection negotiation failed: client connection to 127.12.234.193:32801: connect: Connection refused (error 111)
I20260619 18:49:49.368893 28018 ts_tablet_manager.cc:1918] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:49:49.369006 28018 log.cc:1196] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/8a343af846674f178514fc885509b842
I20260619 18:49:49.369338 28018 ts_tablet_manager.cc:1939] T 8a343af846674f178514fc885509b842 P 4e037a2bcf6a4b93bc4a5da228b98d30: Deleting consensus metadata
I20260619 18:49:49.370785 27512 catalog_manager.cc:4841] TS 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405): tablet 8a343af846674f178514fc885509b842 (table test-table [id=76303ed666bb4227a3db8f5398d58ccd]) successfully deleted
I20260619 18:49:49.371711 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 27858
I20260619 18:49:49.376804 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 27494
I20260619 18:49:49.382517 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46765
--webserver_interface=127.12.234.254
--webserver_port=37727
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:46765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:49:49.496374 28020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:49.496683 28020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:49.496733 28020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:49.501092 28020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:49:49.501171 28020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:49.501192 28020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:49.501211 28020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:49:49.501226 28020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:49:49.506908 28020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:46765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=37727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:49.508037 28020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:49.509123 28020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:49.517838 28028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:49.517848 28025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:49.517851 28026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:49.518285 28020 server_base.cc:1034] running on GCE node
I20260619 18:49:49.518879 28020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:49.520074 28020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:49.521245 28020 hybrid_clock.cc:648] HybridClock initialized: now 1781894989521227 us; error 45 us; skew 500 ppm
Jun 19 18:49:49 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894989, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:49.524272 28020 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:49:49.525329 28020 webserver.cc:460] Webserver started at http://127.12.234.254:37727/ using document root <none> and password file <none>
I20260619 18:49:49.525933 28020 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:49.526007 28020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:49.530244 28020 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:49:49.532745 28035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:49.534297 28020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:49:49.534432 28020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "ca10f8244b5e4b6a9d775ee293671288"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "91d25e0688cd48c9dc1d0e6b8c2b1a4c"
server_key_iv: "1039074227a0a21ac206e0f2b1afadff"
server_key_version: "encryptionkey@0"
I20260619 18:49:49.535025 28020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:49.549503 28020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:49.552908 28020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:49.553107 28020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:49.560170 28020 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:46765
I20260619 18:49:49.560417 28087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:46765 every 8 connection(s)
I20260619 18:49:49.561335 28020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:49:49.565773 28088 sys_catalog.cc:263] Verifying existing consensus state
I20260619 18:49:49.567561 28088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Bootstrap starting.
I20260619 18:49:49.568936 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 28020
I20260619 18:49:49.569417 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:32801
--local_ip_for_outbound_sockets=127.12.234.193
--tserver_master_addrs=127.12.234.254:46765
--webserver_port=45003
--webserver_interface=127.12.234.193
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:49:49.585561 28088 log.cc:824] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:49.597739 28088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:49:49.598169 28088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288: Bootstrap complete.
I20260619 18:49:49.602275 28088 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:49.602703 28088 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca10f8244b5e4b6a9d775ee293671288, State: Initialized, Role: FOLLOWER
I20260619 18:49:49.603238 28088 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:49.603368 28088 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:49:49.603433 28088 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:49:49.603515 28088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 2 FOLLOWER]: Advancing to term 3
I20260619 18:49:49.605489 28088 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:49.605829 28088 leader_election.cc:302] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca10f8244b5e4b6a9d775ee293671288; no voters:
I20260619 18:49:49.606139 28088 leader_election.cc:288] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 18:49:49.606315 28092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 18:49:49.606552 28092 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [term 3 LEADER]: Becoming Leader. State: Replica: ca10f8244b5e4b6a9d775ee293671288, State: Running, Role: LEADER
I20260619 18:49:49.606874 28092 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } }
I20260619 18:49:49.607340 28088 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:49:49.607930 28092 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca10f8244b5e4b6a9d775ee293671288. Latest consensus state: current_term: 3 leader_uuid: "ca10f8244b5e4b6a9d775ee293671288" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } } }
I20260619 18:49:49.608045 28092 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:49.608326 28093 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ca10f8244b5e4b6a9d775ee293671288" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca10f8244b5e4b6a9d775ee293671288" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46765 } } }
I20260619 18:49:49.608418 28093 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca10f8244b5e4b6a9d775ee293671288 [sys.catalog]: This master's current role is: LEADER
I20260619 18:49:49.608479 28098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:49:49.613845 28098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:49:49.614558 28098 catalog_manager.cc:1259] Loaded cluster ID: 63648c854de848c49a3ef21cc9c796b6
I20260619 18:49:49.614622 28098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 18:49:49.615590 28098 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0734e7b9ad google::LogMessage::Fail() at ??:0
@ 0x7f0734e7e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0734e7b4de google::LogMessage::Flush() at ??:0
@ 0x7f0734e7ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f073f7bafdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f073f7bbd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f073f7b8579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f073f7fc640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f07370cd3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f07370cdc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f07370cf3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f07370be512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f07387026db start_thread at ??:0
@ 0x7f073397961f clone at ??:0
W20260619 18:49:49.718452 28090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:49.718869 28090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:49.718938 28090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:49.724275 28090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:49.724357 28090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:49.724447 28090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:49:49.730526 28090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:32801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=45003
--tserver_master_addrs=127.12.234.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:49.731663 28090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:49.732784 28090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:49:49.742442 28090 server_base.cc:1034] running on GCE node
W20260619 18:49:49.742403 28129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:49.742405 28130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:49.742602 28132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:49.743032 28090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:49.743711 28090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:49.744901 28090 hybrid_clock.cc:648] HybridClock initialized: now 1781894989744884 us; error 29 us; skew 500 ppm
Jun 19 18:49:49 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894989, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:49.747767 28090 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:49:49.748732 28090 webserver.cc:460] Webserver started at http://127.12.234.193:45003/ using document root <none> and password file <none>
I20260619 18:49:49.749276 28090 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:49.749336 28090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:49.753417 28090 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:49:49.755820 28139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:49.757230 28090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:49:49.757354 28090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "2a5649f03b3e485bb59f4ddccff6834a"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "381461cfdcc55f7101baad2232611fbe"
server_key_iv: "f0b0457ce212862b596f91af252aa48f"
server_key_version: "encryptionkey@0"
I20260619 18:49:49.757975 28090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:49.778620 28090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:49.782161 28090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:49.782393 28090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:49.783011 28090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:49.784163 28146 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 18:49:49.790005 28090 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 18:49:49.790105 28090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260619 18:49:49.790230 28090 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 18:49:49.792027 28090 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 18:49:49.792083 28090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 18:49:49.792426 28146 tablet_bootstrap.cc:492] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Bootstrap starting.
I20260619 18:49:49.801800 28090 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:32801
I20260619 18:49:49.801862 28253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:32801 every 8 connection(s)
I20260619 18:49:49.802872 28090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:49:49.806814 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 28090
I20260619 18:49:49.807219 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:42583
--local_ip_for_outbound_sockets=127.12.234.194
--tserver_master_addrs=127.12.234.254:46765
--webserver_port=36313
--webserver_interface=127.12.234.194
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:49:49.814565 28146 log.cc:824] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Log is configured to *not* fsync() on all Append() calls
I20260619 18:49:49.824177 28146 tablet_bootstrap.cc:492] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:49:49.824602 28146 tablet_bootstrap.cc:492] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Bootstrap complete.
I20260619 18:49:49.825932 28146 ts_tablet_manager.cc:1397] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.004s
I20260619 18:49:49.829519 28146 raft_consensus.cc:348] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.830260 28146 raft_consensus.cc:729] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5649f03b3e485bb59f4ddccff6834a, State: Initialized, Role: FOLLOWER
I20260619 18:49:49.830854 28146 consensus_queue.cc:260] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:49.832027 28146 ts_tablet_manager.cc:1428] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a: Time spent starting tablet: real 0.006s user 0.001s sys 0.008s
W20260619 18:49:49.881567 28256 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:49:49.804864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:45909)
0619 18:49:49.805419 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.805475 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:49.806357 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:49.880558 (+ 74201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":132,"threads_started":1}
W20260619 18:49:49.882128 28254 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:46765 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.254:46765: Client connection negotiation failed: client connection to 127.12.234.254:46765: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 18:49:49.882984 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.882694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:40213)
0619 18:49:49.882835 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.882915 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 18:49:49.883644 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:49.883434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:38671)
0619 18:49:49.883534 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:49.883585 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 18:49:49.883800 28254 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 18:49:49.923629 28257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:49.923995 28257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:49.924059 28257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:49.928532 28257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:49.928629 28257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:49.928730 28257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:49:49.935706 28257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:42583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=36313
--tserver_master_addrs=127.12.234.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:49.936879 28257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:49.937951 28257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:49.948139 28264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:49.948267 28257 server_base.cc:1034] running on GCE node
W20260619 18:49:49.948303 28267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:49.948139 28265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:49.948787 28257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:49.949561 28257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:49.950718 28257 hybrid_clock.cc:648] HybridClock initialized: now 1781894989950698 us; error 35 us; skew 500 ppm
Jun 19 18:49:49 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894989, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:49.953471 28257 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:49:49.954491 28257 webserver.cc:460] Webserver started at http://127.12.234.194:36313/ using document root <none> and password file <none>
I20260619 18:49:49.955061 28257 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:49.955155 28257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:49.959287 28257 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:49:49.961474 28274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:49.962864 28257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:49:49.962984 28257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "3d144ccf030c444f95a120f25d3123ae"
format_stamp: "Formatted at 2026-06-19 18:49:47 on dist-test-slave-fljv"
server_key: "fc40cf871a8a6352f74ba1f56d088090"
server_key_iv: "588b7cddff268a534c08820d07598c2b"
server_key_version: "encryptionkey@0"
I20260619 18:49:49.963572 28257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:49.997866 28257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:50.001212 28257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:50.001401 28257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:50.002000 28257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:50.002938 28257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:50.002985 28257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:50.003041 28257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:50.003070 28257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:50.012064 28257 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:42583
I20260619 18:49:50.012089 28387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:42583 every 8 connection(s)
I20260619 18:49:50.013159 28257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:49:50.015122 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 28257
I20260619 18:49:50.015530 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:37405
--local_ip_for_outbound_sockets=127.12.234.195
--tserver_master_addrs=127.12.234.254:46765
--webserver_port=33409
--webserver_interface=127.12.234.195
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:49:50.017273 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:50.015551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:53039)
0619 18:49:50.016118 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:50.016261 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":144,"threads_started":1}
W20260619 18:49:50.017735 28388 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:46765 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.254:46765: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
I20260619 18:49:50.018384 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:50.018105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:36917)
0619 18:49:50.018260 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:50.018319 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 18:49:50.022971 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:50.022657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:45831)
0619 18:49:50.022813 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:50.022879 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260619 18:49:50.023192 28388 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 18:49:50.129276 28391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:49:50.129587 28391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:49:50.129633 28391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:49:50.133841 28391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:49:50.133913 28391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:49:50.133976 28391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:49:50.139478 28391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:37405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=33409
--tserver_master_addrs=127.12.234.254:46765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:49:50.140537 28391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:49:50.141523 28391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:49:50.151631 28396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:50.151679 28391 server_base.cc:1034] running on GCE node
W20260619 18:49:50.151572 28399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:49:50.151574 28397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:49:50.152707 28391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:49:50.153398 28391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:49:50.154569 28391 hybrid_clock.cc:648] HybridClock initialized: now 1781894990154549 us; error 35 us; skew 500 ppm
Jun 19 18:49:50 dist-test-slave-fljv krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894990, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:49:50.157433 28391 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:49:50.158509 28391 webserver.cc:460] Webserver started at http://127.12.234.195:33409/ using document root <none> and password file <none>
I20260619 18:49:50.159099 28391 fs_manager.cc:359] Metadata directory not provided
I20260619 18:49:50.159168 28391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:49:50.163305 28391 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:49:50.165524 28406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:50.166924 28391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:49:50.167048 28391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "4e037a2bcf6a4b93bc4a5da228b98d30"
format_stamp: "Formatted at 2026-06-19 18:49:48 on dist-test-slave-fljv"
server_key: "2b2c61428031eb334206ce9cf2377daa"
server_key_iv: "d00587a7715682580f0299e3736ba852"
server_key_version: "encryptionkey@0"
I20260619 18:49:50.167618 28391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:49:50.179762 28391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:49:50.183137 28391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:49:50.183326 28391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:49:50.183952 28391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:49:50.184790 28391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:49:50.184837 28391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:50.184880 28391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:49:50.184911 28391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:49:50.194115 28391 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:37405
I20260619 18:49:50.194151 28519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:37405 every 8 connection(s)
I20260619 18:49:50.195209 28391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:49:50.199287 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:50.197509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:49109)
0619 18:49:50.198018 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:50.198522 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":113,"threads_started":1}
W20260619 18:49:50.199707 28520 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:46765 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.254:46765: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
I20260619 18:49:50.200342 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:50.200132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:50125)
0619 18:49:50.200235 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:50.200285 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 18:49:50.200913 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:50.200722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:59625)
0619 18:49:50.200821 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:50.200866 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 18:49:50.201067 28520 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 18:49:50.202520 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 28391
I20260619 18:49:50.884963 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:50.884485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:37171)
0619 18:49:50.884774 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:50.884844 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":85,"threads_started":1}
I20260619 18:49:51.024384 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:51.023877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:48697)
0619 18:49:51.024190 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:51.024257 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":77,"threads_started":1}
I20260619 18:49:51.202270 28526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:51.201753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:55513)
0619 18:49:51.202043 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:51.202111 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
I20260619 18:49:51.284202 28527 raft_consensus.cc:482] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:49:51.284397 28527 raft_consensus.cc:504] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:51.285682 28527 leader_election.cc:288] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405), 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583)
Jun 19 18:49:51 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894989, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.195@KRBTEST.COM
Jun 19 18:49:51 dist-test-slave-fljv krb5kdc[26945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781894989, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.194@KRBTEST.COM
I20260619 18:49:51.303007 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:51.285912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:42583 (local address 127.12.234.193:46007)
0619 18:49:51.286287 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:51.286300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:51.286395 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:51.288401 (+ 2006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:51.288408 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:51.289051 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:51.289875 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:51.289898 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:51.291918 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:51.291922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:51.293514 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:51.293527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:51.294591 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:51.294595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:51.294866 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:51.295570 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:51.295598 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:51.297949 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:51.300269 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:51.300272 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:51.300274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:51.300475 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:51.300923 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:51.300926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:51.300928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:51.301023 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:51.301799 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:51.301802 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:51.302214 (+ 412us) client_negotiation.cc:769] Sending connection context
0619 18:49:51.302442 (+ 228us) client_negotiation.cc:240] Negotiation successful
0619 18:49:51.302864 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":485,"spinlock_wait_cycles":26240,"thread_start_us":92,"threads_started":1}
I20260619 18:49:51.303241 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:51.286235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:46007 (local address 127.12.234.194:42583)
0619 18:49:51.286627 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:51.286652 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:51.286688 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:49:51.287439 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:51.287447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:51.287914 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:51.288257 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:51.290093 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:51.291740 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:51.293654 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:51.294465 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:51.294639 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:51.298204 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:51.298213 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:51.298231 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:51.298270 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:51.300156 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:51.300575 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:51.300580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:51.300585 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:51.300778 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:51.301103 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:51.301106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:51.301108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:51.301640 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:51.301809 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:51.302613 (+ 804us) server_negotiation.cc:299] Negotiation successful
0619 18:49:51.303045 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":74,"threads_started":1}
I20260619 18:49:51.303241 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:51.286030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:35963 (local address 127.12.234.195:37405)
0619 18:49:51.286353 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0619 18:49:51.286379 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0619 18:49:51.286425 (+ 46us) server_negotiation.cc:407] Connection header received
0619 18:49:51.287440 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:49:51.287447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:49:51.287914 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:49:51.288257 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:49:51.290094 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:51.291740 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:51.293654 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:49:51.294464 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:49:51.294639 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:51.297856 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:49:51.297861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:49:51.297876 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:49:51.297909 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:49:51.299852 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:51.300433 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:51.300438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:51.300443 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:51.300588 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:49:51.301064 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:49:51.301068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:49:51.301069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:49:51.301640 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:49:51.301772 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 18:49:51.302613 (+ 841us) server_negotiation.cc:299] Negotiation successful
0619 18:49:51.303047 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":114,"threads_started":1}
I20260619 18:49:51.303007 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:51.285843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:37405 (local address 127.12.234.193:35963)
0619 18:49:51.285951 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:51.285965 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:49:51.286067 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:49:51.288401 (+ 2334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:49:51.288408 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:49:51.289051 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:49:51.289875 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:51.289898 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:51.291918 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:51.291922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:51.293514 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:49:51.293527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:49:51.294591 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:49:51.294595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:49:51.294865 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:49:51.295593 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:49:51.295605 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:49:51.297566 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:49:51.299975 (+ 2409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:51.299981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:51.299994 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:51.300312 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:51.300762 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:49:51.300765 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:49:51.300766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:49:51.300967 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:49:51.301788 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:49:51.301795 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:49:51.302218 (+ 423us) client_negotiation.cc:769] Sending connection context
0619 18:49:51.302442 (+ 224us) client_negotiation.cc:240] Negotiation successful
0619 18:49:51.302798 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":284,"spinlock_wait_cycles":23040}
W20260619 18:49:51.305807 28142 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583): Not found: Tablet not found: 8a343af846674f178514fc885509b842
W20260619 18:49:51.306382 28141 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405): Not found: Tablet not found: 8a343af846674f178514fc885509b842
I20260619 18:49:51.306483 28141 leader_election.cc:302] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a5649f03b3e485bb59f4ddccff6834a; no voters: 3d144ccf030c444f95a120f25d3123ae, 4e037a2bcf6a4b93bc4a5da228b98d30
I20260619 18:49:51.306756 28527 raft_consensus.cc:2726] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:49:51.886365 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:51.885863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:37029)
0619 18:49:51.886161 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:51.886267 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":75,"threads_started":1}
I20260619 18:49:52.025630 28532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:52.025194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:43395)
0619 18:49:52.025468 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:52.025531 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
I20260619 18:49:52.203503 28533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:52.203068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:48003)
0619 18:49:52.203339 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:52.203405 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260619 18:49:52.887596 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:52.887137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:55945)
0619 18:49:52.887435 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:52.887498 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":97,"threads_started":1}
I20260619 18:49:53.026889 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:53.026445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:33859)
0619 18:49:53.026713 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:53.026775 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":73,"threads_started":1}
I20260619 18:49:53.041121 28536 raft_consensus.cc:482] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:49:53.041257 28536 raft_consensus.cc:504] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:53.041679 28536 leader_election.cc:288] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405), 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583)
W20260619 18:49:53.042550 28141 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405): Not found: Tablet not found: 8a343af846674f178514fc885509b842
W20260619 18:49:53.042646 28142 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583): Not found: Tablet not found: 8a343af846674f178514fc885509b842
I20260619 18:49:53.042797 28142 leader_election.cc:302] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a5649f03b3e485bb59f4ddccff6834a; no voters: 3d144ccf030c444f95a120f25d3123ae, 4e037a2bcf6a4b93bc4a5da228b98d30
I20260619 18:49:53.042959 28536 raft_consensus.cc:2726] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:49:53.204727 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:53.204268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:33963)
0619 18:49:53.204543 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:53.204617 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":88,"threads_started":1}
I20260619 18:49:53.888804 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:53.888384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:58303)
0619 18:49:53.888631 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:53.888696 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260619 18:49:54.028328 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:54.027858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:44229)
0619 18:49:54.028145 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:54.028213 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":89,"threads_started":1}
I20260619 18:49:54.205929 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:54.205493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:60689)
0619 18:49:54.205771 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:54.205832 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260619 18:49:54.709900 28541 raft_consensus.cc:482] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:49:54.710044 28541 raft_consensus.cc:504] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:54.710549 28541 leader_election.cc:288] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405), 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583)
W20260619 18:49:54.711401 28141 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405): Not found: Tablet not found: 8a343af846674f178514fc885509b842
W20260619 18:49:54.711526 28142 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583): Not found: Tablet not found: 8a343af846674f178514fc885509b842
I20260619 18:49:54.711585 28142 leader_election.cc:302] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a5649f03b3e485bb59f4ddccff6834a; no voters: 3d144ccf030c444f95a120f25d3123ae, 4e037a2bcf6a4b93bc4a5da228b98d30
I20260619 18:49:54.711721 28541 raft_consensus.cc:2726] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:49:54.890314 28542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:54.889734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:51083)
0619 18:49:54.890105 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:54.890167 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":85,"threads_started":1}
I20260619 18:49:55.029723 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:55.029253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:43061)
0619 18:49:55.029551 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:55.029622 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":75,"threads_started":1}
I20260619 18:49:55.207221 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:55.206828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:56917)
0619 18:49:55.207073 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:55.207124 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":87,"threads_started":1}
I20260619 18:49:55.891470 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:55.891043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:49621)
0619 18:49:55.891297 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:55.891368 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":71,"threads_started":1}
I20260619 18:49:56.031112 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:56.030565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:49193)
0619 18:49:56.030920 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:56.030990 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":80,"threads_started":1}
I20260619 18:49:56.208580 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:56.208062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:43481)
0619 18:49:56.208366 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:56.208458 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":89,"threads_started":1}
I20260619 18:49:56.538836 28548 raft_consensus.cc:482] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:49:56.538990 28548 raft_consensus.cc:504] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:49:56.539462 28548 leader_election.cc:288] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405), 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583)
W20260619 18:49:56.540222 28141 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405): Not found: Tablet not found: 8a343af846674f178514fc885509b842
W20260619 18:49:56.540339 28142 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583): Not found: Tablet not found: 8a343af846674f178514fc885509b842
I20260619 18:49:56.540427 28142 leader_election.cc:302] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a5649f03b3e485bb59f4ddccff6834a; no voters: 3d144ccf030c444f95a120f25d3123ae, 4e037a2bcf6a4b93bc4a5da228b98d30
I20260619 18:49:56.540555 28548 raft_consensus.cc:2726] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:49:56.892575 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:56.892171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:39795)
0619 18:49:56.892424 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:56.892479 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
I20260619 18:49:57.032645 28550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:57.032135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:49305)
0619 18:49:57.032471 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:57.032543 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":75,"threads_started":1}
I20260619 18:49:57.210062 28551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:57.209447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:41971)
0619 18:49:57.209878 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:57.209953 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":90,"threads_started":1}
I20260619 18:49:57.893956 28552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:57.893419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:55271)
0619 18:49:57.893727 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:57.893849 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":77,"threads_started":1}
I20260619 18:49:58.034227 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:58.033684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:49925)
0619 18:49:58.034013 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:58.034085 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
I20260619 18:49:58.211341 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:58.210883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:33385)
0619 18:49:58.211166 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:58.211232 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":86,"threads_started":1}
I20260619 18:49:58.895259 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:58.894803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:59257)
0619 18:49:58.895096 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:58.895159 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":74,"threads_started":1}
I20260619 18:49:59.035501 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:59.035045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:47141)
0619 18:49:59.035344 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:59.035402 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":83,"threads_started":1}
I20260619 18:49:59.212637 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:59.212194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:54701)
0619 18:49:59.212445 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:59.212503 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260619 18:49:59.896596 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:49:59.896131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:54925)
0619 18:49:59.896439 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 18:49:59.896499 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":75,"threads_started":1}
I20260619 18:50:00.036892 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:00.036395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:48815)
0619 18:50:00.036710 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:00.036784 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":78,"threads_started":1}
I20260619 18:50:00.213987 28560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:00.213542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:46449)
0619 18:50:00.213819 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:00.213888 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":91,"threads_started":1}
I20260619 18:50:00.897995 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:00.897503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:35113)
0619 18:50:00.897799 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:00.897868 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":91,"threads_started":1}
I20260619 18:50:01.038483 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:01.037934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:56123)
0619 18:50:01.038283 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:01.038358 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":96,"threads_started":1}
I20260619 18:50:01.215237 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:01.214805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:42681)
0619 18:50:01.215066 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:01.215139 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260619 18:50:01.899276 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:01.898820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:57345)
0619 18:50:01.899110 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:01.899177 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":97,"threads_started":1}
I20260619 18:50:02.039866 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:02.039386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:57561)
0619 18:50:02.039688 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:02.039757 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
I20260619 18:50:02.216593 28566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:02.216067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:54191)
0619 18:50:02.216409 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:02.216488 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":98,"threads_started":1}
I20260619 18:50:02.900580 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:02.900124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:47053)
0619 18:50:02.900408 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:02.900483 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":89,"threads_started":1}
I20260619 18:50:02.942975 28568 raft_consensus.cc:482] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:50:02.943125 28568 raft_consensus.cc:504] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e037a2bcf6a4b93bc4a5da228b98d30" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 37405 } } peers { permanent_uuid: "2a5649f03b3e485bb59f4ddccff6834a" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 32801 } } peers { permanent_uuid: "3d144ccf030c444f95a120f25d3123ae" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 42583 } }
I20260619 18:50:02.943547 28568 leader_election.cc:288] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405), 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583)
W20260619 18:50:02.944322 28142 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d144ccf030c444f95a120f25d3123ae (127.12.234.194:42583): Not found: Tablet not found: 8a343af846674f178514fc885509b842
W20260619 18:50:02.944447 28141 leader_election.cc:341] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e037a2bcf6a4b93bc4a5da228b98d30 (127.12.234.195:37405): Not found: Tablet not found: 8a343af846674f178514fc885509b842
I20260619 18:50:02.944545 28141 leader_election.cc:302] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a5649f03b3e485bb59f4ddccff6834a; no voters: 3d144ccf030c444f95a120f25d3123ae, 4e037a2bcf6a4b93bc4a5da228b98d30
I20260619 18:50:02.944702 28568 raft_consensus.cc:2726] T 8a343af846674f178514fc885509b842 P 2a5649f03b3e485bb59f4ddccff6834a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:50:03.041181 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:03.040702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:53163)
0619 18:50:03.041022 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:03.041084 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":79,"threads_started":1}
I20260619 18:50:03.217960 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:03.217498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:32941)
0619 18:50:03.217802 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:03.217862 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":118,"threads_started":1}
I20260619 18:50:03.901876 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:03.901410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:35639)
0619 18:50:03.901706 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:03.901776 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":100,"threads_started":1}
I20260619 18:50:04.042627 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:04.042133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:50645)
0619 18:50:04.042456 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:04.042528 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":109,"threads_started":1}
I20260619 18:50:04.219218 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:04.218771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.195:38185)
0619 18:50:04.219051 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:04.219115 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":89,"threads_started":1}
I20260619 18:50:04.903478 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:04.903000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.193:48859)
0619 18:50:04.903299 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:04.903375 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":75,"threads_started":1}
I20260619 18:50:05.044094 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:05.043587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:46765 (local address 127.12.234.194:60619)
0619 18:50:05.043898 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:05.043975 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:46765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":78,"threads_started":1}
I20260619 18:50:05.203400 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 28090
I20260619 18:50:05.208580 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 28257
I20260619 18:50:05.213573 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 28391
2026-06-19T18:50:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18944 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:50:05 dist-test-slave-fljv krb5kdc[28578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:50:05 dist-test-slave-fljv krb5kdc[28578](info): set up 2 sockets
Jun 19 18:50:05 dist-test-slave-fljv krb5kdc[28578](info): commencing operation
krb5kdc: starting...
W20260619 18:50:06.190279 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:50:06 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:50:06.313908 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35779
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:35779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:50:06.425143 28591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:06.425464 28591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:06.425527 28591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:06.429755 28591 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 18:50:06.429833 28591 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 18:50:06.429855 28591 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 18:50:06.429875 28591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:50:06.429900 28591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:06.429926 28591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:06.430018 28591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:50:06.430051 28591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:50:06.435772 28591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:35779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:06.436951 28591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:06.438022 28591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:06.446671 28599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:06.446671 28596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:06.446683 28597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:06.447070 28591 server_base.cc:1034] running on GCE node
I20260619 18:50:06.447633 28591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:06.448812 28591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:06.450013 28591 hybrid_clock.cc:648] HybridClock initialized: now 1781895006449985 us; error 42 us; skew 500 ppm
Jun 19 18:50:06 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:06.452711 28591 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:50:06.453986 28591 webserver.cc:460] Webserver started at http://127.12.234.254:40985/ using document root <none> and password file <none>
I20260619 18:50:06.454617 28591 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:06.454691 28591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:06.454919 28591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:50:06.457111 28591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "bd7b152b33344b5b9c93fb7517152aed"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "45dc860dbc2ef8e44c528d656727e58d"
server_key_iv: "135915d9f66ba615d2f0ce506019403b"
server_key_version: "encryptionkey@0"
I20260619 18:50:06.457707 28591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "bd7b152b33344b5b9c93fb7517152aed"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "45dc860dbc2ef8e44c528d656727e58d"
server_key_iv: "135915d9f66ba615d2f0ce506019403b"
server_key_version: "encryptionkey@0"
I20260619 18:50:06.461953 28591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260619 18:50:06.464720 28606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:06.466161 28591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:50:06.466373 28591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "bd7b152b33344b5b9c93fb7517152aed"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "45dc860dbc2ef8e44c528d656727e58d"
server_key_iv: "135915d9f66ba615d2f0ce506019403b"
server_key_version: "encryptionkey@0"
I20260619 18:50:06.466492 28591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:06.501525 28591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:06.504688 28591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:06.504894 28591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:06.511904 28591 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:35779
I20260619 18:50:06.512188 28658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:35779 every 8 connection(s)
I20260619 18:50:06.512995 28591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:50:06.516155 28659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:50:06.520831 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 28591
I20260619 18:50:06.520977 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:50:06.521267 13227 external_mini_cluster.cc:1428] Setting key 6ff6ac279604d2ce6678a74f4d0dcfa7
I20260619 18:50:06.522809 28659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Bootstrap starting.
I20260619 18:50:06.525395 28659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Neither blocks nor log segments found. Creating new log.
I20260619 18:50:06.526265 28659 log.cc:824] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Log is configured to *not* fsync() on all Append() calls
I20260619 18:50:06.528661 28659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: No bootstrap required, opened a new log
Jun 19 18:50:06 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:50:06.533032 28659 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:06.533344 28659 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:50:06.533435 28659 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd7b152b33344b5b9c93fb7517152aed, State: Initialized, Role: FOLLOWER
I20260619 18:50:06.533937 28659 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:06.534081 28659 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:50:06.534150 28659 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:50:06.534289 28659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:50:06.535584 28659 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:06.535990 28659 leader_election.cc:302] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd7b152b33344b5b9c93fb7517152aed; no voters:
I20260619 18:50:06.536358 28659 leader_election.cc:288] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:50:06.536473 28664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:50:06.536705 28664 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 1 LEADER]: Becoming Leader. State: Replica: bd7b152b33344b5b9c93fb7517152aed, State: Running, Role: LEADER
I20260619 18:50:06.537087 28664 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:06.537154 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:06.522788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.12.234.254:35779)
0619 18:50:06.523246 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:06.523263 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:06.523300 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:50:06.524007 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:06.524026 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:06.524379 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:06.524677 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:06.525609 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:06.526723 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:06.527717 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:06.528442 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:06.528619 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:06.531395 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:06.531404 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:06.531426 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:06.531467 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:06.533558 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:06.534102 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:06.534108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:06.534115 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:06.534322 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:06.534641 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:06.534645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:06.534647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:06.535200 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:06.535420 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:06.535722 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 18:50:06.536064 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":139,"threads_started":1}
I20260619 18:50:06.537683 28659 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:50:06.538800 28666 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd7b152b33344b5b9c93fb7517152aed. Latest consensus state: current_term: 1 leader_uuid: "bd7b152b33344b5b9c93fb7517152aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } } }
I20260619 18:50:06.538926 28666 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: This master's current role is: LEADER
I20260619 18:50:06.539214 28665 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd7b152b33344b5b9c93fb7517152aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } } }
I20260619 18:50:06.539314 28665 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: This master's current role is: LEADER
I20260619 18:50:06.540125 28673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:50:06.542832 28673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:50:06.549422 28673 catalog_manager.cc:1347] Generated new cluster ID: fda11037b1df4de2883ea8009b8646dd
I20260619 18:50:06.549527 28673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:50:06.561514 28673 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:50:06.562602 28673 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:50:06.569669 28673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Generated new TSK 0
I20260619 18:50:06.570654 28673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:50:06.605129 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35779
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:50:06.722170 28685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:06.722500 28685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:06.722548 28685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:06.726717 28685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:06.726802 28685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:06.726874 28685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:50:06.732756 28685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:06.733971 28685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:06.735064 28685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:06.745188 28690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:06.745200 28693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:06.745230 28685 server_base.cc:1034] running on GCE node
W20260619 18:50:06.745190 28691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:06.745862 28685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:06.746585 28685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:06.747756 28685 hybrid_clock.cc:648] HybridClock initialized: now 1781895006747734 us; error 36 us; skew 500 ppm
Jun 19 18:50:06 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:06.750631 28685 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:50:06.751634 28685 webserver.cc:460] Webserver started at http://127.12.234.193:34469/ using document root <none> and password file <none>
I20260619 18:50:06.752214 28685 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:06.752281 28685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:06.752485 28685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:50:06.754642 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "fec2848dcfa7284b4ea570f3069e1043"
server_key_iv: "c4fa37caf2f9644b1e06af857deb1313"
server_key_version: "encryptionkey@0"
I20260619 18:50:06.755194 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "fec2848dcfa7284b4ea570f3069e1043"
server_key_iv: "c4fa37caf2f9644b1e06af857deb1313"
server_key_version: "encryptionkey@0"
I20260619 18:50:06.759795 28685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:50:06.762598 28700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:06.763863 28685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:50:06.763978 28685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "fec2848dcfa7284b4ea570f3069e1043"
server_key_iv: "c4fa37caf2f9644b1e06af857deb1313"
server_key_version: "encryptionkey@0"
I20260619 18:50:06.764107 28685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:06.778473 28685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:06.781795 28685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:06.781989 28685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:06.782657 28685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:50:06.783547 28685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:50:06.783601 28685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:06.783659 28685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:50:06.783687 28685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:06.792855 28685 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:37799
I20260619 18:50:06.792930 28813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:37799 every 8 connection(s)
I20260619 18:50:06.793814 28685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:50:06.802028 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 28685
I20260619 18:50:06.802145 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:50:06.802513 13227 external_mini_cluster.cc:1428] Setting key d4e8aea7e58d0261648f5ad92cb43a69
Jun 19 18:50:06 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:50:06.807885 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:06.795959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:50071 (local address 127.12.234.254:35779)
0619 18:50:06.796115 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:06.796119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:06.797166 (+ 1047us) server_negotiation.cc:407] Connection header received
0619 18:50:06.798058 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:06.798061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:06.798111 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:06.798224 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:06.799427 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:06.800006 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:06.801139 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:06.801375 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:06.801463 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:06.804281 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:06.804285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:06.804287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:06.804312 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:06.805736 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:06.806377 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:06.806381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:06.806383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:06.806474 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:06.806798 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:06.806802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:06.806804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:06.806992 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:06.807078 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:06.807596 (+ 518us) server_negotiation.cc:299] Negotiation successful
0619 18:50:06.807723 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 18:50:06.808772 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:06.796342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:50071)
0619 18:50:06.797008 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:06.797050 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:06.797840 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:06.798379 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:06.798386 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:06.798774 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:50:06.799273 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:06.799288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:06.800114 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:06.800116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:06.801034 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:06.801043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:06.801517 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:06.801520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:06.801669 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:06.802353 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:50:06.802377 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:50:06.804120 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:50:06.805856 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:06.805864 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:06.805883 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:06.806286 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:06.806558 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:06.806561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:06.806563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:06.806713 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:06.807090 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:50:06.807097 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:50:06.807367 (+ 270us) client_negotiation.cc:769] Sending connection context
0619 18:50:06.807542 (+ 175us) client_negotiation.cc:240] Negotiation successful
0619 18:50:06.807898 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":125,"threads_started":1}
I20260619 18:50:06.810056 28814 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35779
I20260619 18:50:06.810317 28814 heartbeater.cc:464] Registering TS with master...
I20260619 18:50:06.810822 28814 heartbeater.cc:511] Master 127.12.234.254:35779 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:50:06.812350 28623 ts_manager.cc:194] Registered new tserver with Master: a3c8ff331acc44d8ad159bd0ae36fa0f (127.12.234.193:37799)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:50:06.813571 28623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:50071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:50:06.828665 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35779
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:50:06.940793 28819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:06.941078 28819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:06.941124 28819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:06.945323 28819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:06.945398 28819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:06.945470 28819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:50:06.950894 28819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:06.952013 28819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:06.953075 28819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:06.962883 28824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:06.962903 28827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:06.963052 28819 server_base.cc:1034] running on GCE node
W20260619 18:50:06.962905 28825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:06.963562 28819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:06.964314 28819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:06.965539 28819 hybrid_clock.cc:648] HybridClock initialized: now 1781895006965501 us; error 51 us; skew 500 ppm
Jun 19 18:50:06 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:06.968369 28819 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:50:06.969368 28819 webserver.cc:460] Webserver started at http://127.12.234.194:36025/ using document root <none> and password file <none>
I20260619 18:50:06.969919 28819 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:06.969985 28819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:06.970228 28819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:50:06.972362 28819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "e711fc080b1c4dfbb6ca86a84722e25b"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "aee7442d1b3cba728d4fe76ff0c0b20f"
server_key_iv: "5742c407a70ba73d0a2fe35f3cb6cbf5"
server_key_version: "encryptionkey@0"
I20260619 18:50:06.972911 28819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "e711fc080b1c4dfbb6ca86a84722e25b"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "aee7442d1b3cba728d4fe76ff0c0b20f"
server_key_iv: "5742c407a70ba73d0a2fe35f3cb6cbf5"
server_key_version: "encryptionkey@0"
I20260619 18:50:06.976753 28819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:50:06.979542 28834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:06.981016 28819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:50:06.981146 28819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "e711fc080b1c4dfbb6ca86a84722e25b"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "aee7442d1b3cba728d4fe76ff0c0b20f"
server_key_iv: "5742c407a70ba73d0a2fe35f3cb6cbf5"
server_key_version: "encryptionkey@0"
I20260619 18:50:06.981263 28819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:07.003255 28819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:07.006443 28819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:07.006629 28819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:07.007278 28819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:50:07.008131 28819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:50:07.008193 28819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:07.008244 28819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:50:07.008266 28819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:07.016944 28819 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:36703
I20260619 18:50:07.017043 28947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:36703 every 8 connection(s)
I20260619 18:50:07.017935 28819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:50:07.025194 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 28819
I20260619 18:50:07.025318 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:50:07.025655 13227 external_mini_cluster.cc:1428] Setting key 84cd6e0731169058a765cd45daea9825
Jun 19 18:50:07 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:50:07.032182 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.019883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:37433 (local address 127.12.234.254:35779)
0619 18:50:07.020014 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:07.020018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:07.021105 (+ 1087us) server_negotiation.cc:407] Connection header received
0619 18:50:07.022042 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:07.022045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:07.022101 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:07.022191 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:07.023390 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.023926 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.025089 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.025318 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.025413 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.028308 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:07.028312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:07.028315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:07.028345 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:07.029776 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:07.030397 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:07.030400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:07.030401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:07.030503 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:07.030901 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:07.030905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:07.030907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:07.031119 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:07.031226 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:07.031928 (+ 702us) server_negotiation.cc:299] Negotiation successful
0619 18:50:07.032053 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 18:50:07.033439 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.020272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:37433)
0619 18:50:07.020931 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:07.020975 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:07.021825 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:07.022367 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:07.022394 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:07.022754 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:50:07.023243 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.023257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.024045 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.024048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.024971 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.024981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.025523 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.025528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.025737 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.026535 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:50:07.026555 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:50:07.028131 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:50:07.029900 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:07.029906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:07.029925 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:07.030307 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:07.030582 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:07.030585 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:07.030586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:07.030769 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:07.031235 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:50:07.031244 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:50:07.031652 (+ 408us) client_negotiation.cc:769] Sending connection context
0619 18:50:07.031871 (+ 219us) client_negotiation.cc:240] Negotiation successful
0619 18:50:07.032269 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":117,"threads_started":1}
I20260619 18:50:07.034757 28948 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35779
I20260619 18:50:07.035014 28948 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:50:07.035542 28948 heartbeater.cc:511] Master 127.12.234.254:35779 requested a full tablet report, sending...
I20260619 18:50:07.036607 28623 ts_manager.cc:194] Registered new tserver with Master: e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703)
I20260619 18:50:07.037170 28623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:37433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:50:07.054075 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35779
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:50:07.164834 28953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:07.165114 28953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:07.165158 28953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:07.169337 28953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:07.169400 28953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:07.169476 28953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:50:07.175112 28953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:07.176203 28953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:07.177249 28953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:07.187206 28961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:07.187239 28959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:07.187254 28958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:07.187389 28953 server_base.cc:1034] running on GCE node
I20260619 18:50:07.187815 28953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:07.188452 28953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:07.189568 28953 hybrid_clock.cc:648] HybridClock initialized: now 1781895007189547 us; error 36 us; skew 500 ppm
Jun 19 18:50:07 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895007, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:07.192224 28953 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:50:07.193149 28953 webserver.cc:460] Webserver started at http://127.12.234.195:37461/ using document root <none> and password file <none>
I20260619 18:50:07.193710 28953 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:07.193770 28953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:07.193977 28953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:50:07.196084 28953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "308b15003edc42a48b5f9aecb3fadedb"
format_stamp: "Formatted at 2026-06-19 18:50:07 on dist-test-slave-fljv"
server_key: "30e31d8db5b011ffc166eaf9bf2dfe45"
server_key_iv: "2f229c4e03d1e8bf1f0431b2846cc403"
server_key_version: "encryptionkey@0"
I20260619 18:50:07.196635 28953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "308b15003edc42a48b5f9aecb3fadedb"
format_stamp: "Formatted at 2026-06-19 18:50:07 on dist-test-slave-fljv"
server_key: "30e31d8db5b011ffc166eaf9bf2dfe45"
server_key_iv: "2f229c4e03d1e8bf1f0431b2846cc403"
server_key_version: "encryptionkey@0"
I20260619 18:50:07.200431 28953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:50:07.203015 28968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:07.204417 28953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:50:07.204547 28953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "308b15003edc42a48b5f9aecb3fadedb"
format_stamp: "Formatted at 2026-06-19 18:50:07 on dist-test-slave-fljv"
server_key: "30e31d8db5b011ffc166eaf9bf2dfe45"
server_key_iv: "2f229c4e03d1e8bf1f0431b2846cc403"
server_key_version: "encryptionkey@0"
I20260619 18:50:07.204679 28953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:07.225638 28953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:07.228888 28953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:07.229070 28953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:07.229710 28953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:50:07.230635 28953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:50:07.230682 28953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:07.230723 28953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:50:07.230751 28953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:07.239385 28953 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:33163
I20260619 18:50:07.239413 29081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:33163 every 8 connection(s)
I20260619 18:50:07.240338 28953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:50:07.240530 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 28953
I20260619 18:50:07.240649 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:50:07.240967 13227 external_mini_cluster.cc:1428] Setting key 1ac937a79f9a3bd5eb4cc0d39507d46f
Jun 19 18:50:07 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895007, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:50:07.253819 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.242243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:48907 (local address 127.12.234.254:35779)
0619 18:50:07.242393 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:07.242404 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:07.243762 (+ 1358us) server_negotiation.cc:407] Connection header received
0619 18:50:07.244685 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:07.244689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:07.244744 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:07.244818 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:07.246027 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.246601 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.247740 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.247960 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.248054 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.250504 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:07.250507 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:07.250509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:07.250536 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:07.251790 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:07.252281 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:07.252285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:07.252287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:07.252364 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:07.252690 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:07.252693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:07.252695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:07.252887 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:07.252974 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:07.253488 (+ 514us) server_negotiation.cc:299] Negotiation successful
0619 18:50:07.253650 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 18:50:07.254825 29084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.242834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:48907)
0619 18:50:07.243634 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:07.243678 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:07.244458 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:07.245011 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:07.245019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:07.245382 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:50:07.245884 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.245897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.246719 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.246723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.247626 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.247635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.248107 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.248111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.248280 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.248844 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:50:07.248861 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:50:07.250324 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:50:07.251900 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:07.251906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:07.251923 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:07.252184 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:07.252445 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:07.252448 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:07.252451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:07.252603 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:07.252985 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:50:07.252992 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:50:07.253258 (+ 266us) client_negotiation.cc:769] Sending connection context
0619 18:50:07.253432 (+ 174us) client_negotiation.cc:240] Negotiation successful
0619 18:50:07.253794 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":160,"threads_started":1}
I20260619 18:50:07.255916 29082 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35779
I20260619 18:50:07.256155 29082 heartbeater.cc:464] Registering TS with master...
I20260619 18:50:07.256642 29082 heartbeater.cc:511] Master 127.12.234.254:35779 requested a full tablet report, sending...
I20260619 18:50:07.257668 28623 ts_manager.cc:194] Registered new tserver with Master: 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163)
I20260619 18:50:07.258270 28623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:48907
I20260619 18:50:07.267554 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:50:07.276937 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.269650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49232 (local address 127.12.234.254:35779)
0619 18:50:07.269848 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:07.269852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:07.269938 (+ 86us) server_negotiation.cc:407] Connection header received
0619 18:50:07.270077 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:07.270079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:07.270128 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:07.270242 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:07.270998 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.271566 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.272476 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.272706 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.272845 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.274212 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:07.274216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:07.274219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:07.274253 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:07.275475 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:07.275863 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:07.275865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:07.275866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:07.275942 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:07.276160 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:07.276162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:07.276163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:07.276375 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:07.276474 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:07.276686 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 18:50:07.276781 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 18:50:07.279783 28623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:50:07.282912 28623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:50:07.297467 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.291577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:33163 (local address 127.0.0.1:60736)
0619 18:50:07.291942 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:07.292009 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:07.292138 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:07.292910 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:07.292917 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:07.292942 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:50:07.293080 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.293088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.294475 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.294486 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.295876 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.295887 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.296908 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.296911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.297029 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.297067 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 18:50:07.297173 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 18:50:07.297299 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":22,"thread_start_us":91,"threads_started":1}
I20260619 18:50:07.297670 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.292670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:37799 (local address 127.0.0.1:54360)
0619 18:50:07.292885 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:07.292896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:07.292963 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:07.293903 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:07.293906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:07.293923 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:50:07.293993 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.293998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.295135 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.295138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.296284 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.296294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.297307 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.297311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.297408 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.297424 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 18:50:07.297492 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 18:50:07.297555 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":19,"thread_start_us":90,"threads_started":1}
I20260619 18:50:07.297780 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.293115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54360 (local address 127.12.234.193:37799)
0619 18:50:07.293495 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:07.293509 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:07.293531 (+ 22us) server_negotiation.cc:407] Connection header received
0619 18:50:07.293603 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:07.293607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:07.293790 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:07.293931 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:50:07.294072 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.295007 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.296407 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.297183 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.297372 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.297485 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:07.297529 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 18:50:07.297626 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":115,"threads_started":1}
I20260619 18:50:07.297930 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.291752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.12.234.195:33163)
0619 18:50:07.292107 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:07.292119 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:07.292144 (+ 25us) server_negotiation.cc:407] Connection header received
0619 18:50:07.292288 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:07.292294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:07.292547 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:07.292709 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:50:07.293217 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.294355 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.296012 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.296795 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.296909 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.297060 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:07.297679 (+ 619us) server_negotiation.cc:299] Negotiation successful
0619 18:50:07.297794 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":144,"threads_started":1}
I20260619 18:50:07.298106 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.292338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:36703 (local address 127.0.0.1:40230)
0619 18:50:07.292762 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:07.292775 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:07.292860 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:07.293631 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:07.293634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:07.293650 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:50:07.293736 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.293741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.294978 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.294982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.296152 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.296162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.297214 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.297217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.297299 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.297852 (+ 553us) client_negotiation.cc:769] Sending connection context
0619 18:50:07.297918 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 18:50:07.297989 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":160,"thread_start_us":103,"threads_started":1}
I20260619 18:50:07.298457 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.292512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40230 (local address 127.12.234.194:36703)
0619 18:50:07.293203 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:07.293216 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:07.293238 (+ 22us) server_negotiation.cc:407] Connection header received
0619 18:50:07.293312 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:07.293317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:07.293505 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:07.293638 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:50:07.293840 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.294844 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.296268 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.297088 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.297212 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.297348 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:07.298265 (+ 917us) server_negotiation.cc:299] Negotiation successful
0619 18:50:07.298349 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":111,"threads_started":1}
I20260619 18:50:07.301220 28748 tablet_service.cc:1467] Processing CreateTablet for tablet d5209b7922024b48b23315f10049f3f8 (DEFAULT_TABLE table=test-table [id=a4bb7bbdb07d45be9d276b79904aa52b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:50:07.301220 29016 tablet_service.cc:1467] Processing CreateTablet for tablet d5209b7922024b48b23315f10049f3f8 (DEFAULT_TABLE table=test-table [id=a4bb7bbdb07d45be9d276b79904aa52b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:50:07.302484 28748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5209b7922024b48b23315f10049f3f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:50:07.302488 29016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5209b7922024b48b23315f10049f3f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:50:07.305024 28882 tablet_service.cc:1467] Processing CreateTablet for tablet d5209b7922024b48b23315f10049f3f8 (DEFAULT_TABLE table=test-table [id=a4bb7bbdb07d45be9d276b79904aa52b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:50:07.306290 28882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5209b7922024b48b23315f10049f3f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:50:07.310333 29099 tablet_bootstrap.cc:492] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Bootstrap starting.
I20260619 18:50:07.312369 29100 tablet_bootstrap.cc:492] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: Bootstrap starting.
I20260619 18:50:07.312788 29099 tablet_bootstrap.cc:654] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Neither blocks nor log segments found. Creating new log.
I20260619 18:50:07.313761 29099 log.cc:824] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Log is configured to *not* fsync() on all Append() calls
I20260619 18:50:07.314287 29101 tablet_bootstrap.cc:492] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Bootstrap starting.
I20260619 18:50:07.315187 29100 tablet_bootstrap.cc:654] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: Neither blocks nor log segments found. Creating new log.
I20260619 18:50:07.315953 29100 log.cc:824] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: Log is configured to *not* fsync() on all Append() calls
I20260619 18:50:07.316262 29099 tablet_bootstrap.cc:492] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: No bootstrap required, opened a new log
I20260619 18:50:07.316524 29099 ts_tablet_manager.cc:1397] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:50:07.316828 29101 tablet_bootstrap.cc:654] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Neither blocks nor log segments found. Creating new log.
I20260619 18:50:07.317664 29101 log.cc:824] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Log is configured to *not* fsync() on all Append() calls
I20260619 18:50:07.317751 29100 tablet_bootstrap.cc:492] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: No bootstrap required, opened a new log
I20260619 18:50:07.317926 29100 ts_tablet_manager.cc:1397] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 18:50:07.319944 29101 tablet_bootstrap.cc:492] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: No bootstrap required, opened a new log
I20260619 18:50:07.320225 29101 ts_tablet_manager.cc:1397] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:50:07.321059 29099 raft_consensus.cc:348] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:07.321362 29099 raft_consensus.cc:374] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:50:07.321216 29100 raft_consensus.cc:348] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:07.321447 29099 raft_consensus.cc:729] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c8ff331acc44d8ad159bd0ae36fa0f, State: Initialized, Role: FOLLOWER
I20260619 18:50:07.321511 29100 raft_consensus.cc:374] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:50:07.321570 29100 raft_consensus.cc:729] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308b15003edc42a48b5f9aecb3fadedb, State: Initialized, Role: FOLLOWER
I20260619 18:50:07.321973 29100 consensus_queue.cc:260] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:07.321944 29099 consensus_queue.cc:260] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:07.322683 28814 heartbeater.cc:503] Master 127.12.234.254:35779 was elected leader, sending a full tablet report...
I20260619 18:50:07.323084 29099 ts_tablet_manager.cc:1428] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260619 18:50:07.323971 29100 ts_tablet_manager.cc:1428] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260619 18:50:07.323956 29101 raft_consensus.cc:348] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:07.324281 29101 raft_consensus.cc:374] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:50:07.324311 29082 heartbeater.cc:503] Master 127.12.234.254:35779 was elected leader, sending a full tablet report...
I20260619 18:50:07.324349 29101 raft_consensus.cc:729] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e711fc080b1c4dfbb6ca86a84722e25b, State: Initialized, Role: FOLLOWER
I20260619 18:50:07.324815 29101 consensus_queue.cc:260] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:07.325471 28948 heartbeater.cc:503] Master 127.12.234.254:35779 was elected leader, sending a full tablet report...
I20260619 18:50:07.325867 29101 ts_tablet_manager.cc:1428] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:50:07.337126 29107 raft_consensus.cc:482] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:50:07.337349 29107 raft_consensus.cc:504] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:07.338655 29107 leader_election.cc:288] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163), a3c8ff331acc44d8ad159bd0ae36fa0f (127.12.234.193:37799)
I20260619 18:50:07.343117 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.339065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:57801 (local address 127.12.234.195:33163)
0619 18:50:07.339206 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:07.339211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:07.339244 (+ 33us) server_negotiation.cc:407] Connection header received
0619 18:50:07.339447 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:07.339451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:07.339595 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:07.339683 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:50:07.340003 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.340643 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.341907 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.342529 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.342624 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.342667 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:07.342908 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 18:50:07.343006 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 18:50:07.343137 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.339023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:33163 (local address 127.12.234.194:57801)
0619 18:50:07.339176 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:07.339193 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:07.339284 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:07.339711 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:07.339714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:07.339734 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:50:07.339861 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.339868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.340757 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.340761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.341769 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.341780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.342659 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.342662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.342780 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.342808 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 18:50:07.342899 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 18:50:07.342982 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 18:50:07.343787 29108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.339320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:37799 (local address 127.12.234.194:33097)
0619 18:50:07.339752 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:07.339765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:07.339867 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:07.340223 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:07.340226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:07.340245 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:50:07.340350 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.340356 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.341289 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.341291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.342284 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:07.342296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.343377 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.343381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:07.343489 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.343513 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 18:50:07.343575 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 18:50:07.343643 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":127,"threads_started":1}
I20260619 18:50:07.343786 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.339464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:33097 (local address 127.12.234.193:37799)
0619 18:50:07.339601 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:07.339606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:07.339849 (+ 243us) server_negotiation.cc:407] Connection header received
0619 18:50:07.339970 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:07.339974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:07.340037 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:07.340108 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:50:07.340463 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.341171 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.342459 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.343123 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.343243 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.343293 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:07.343618 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 18:50:07.343688 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 18:50:07.343981 29036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5209b7922024b48b23315f10049f3f8" candidate_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308b15003edc42a48b5f9aecb3fadedb" is_pre_election: true
I20260619 18:50:07.344323 29036 raft_consensus.cc:2454] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e711fc080b1c4dfbb6ca86a84722e25b in term 0.
I20260619 18:50:07.344383 28768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5209b7922024b48b23315f10049f3f8" candidate_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" is_pre_election: true
I20260619 18:50:07.344628 28768 raft_consensus.cc:2454] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e711fc080b1c4dfbb6ca86a84722e25b in term 0.
I20260619 18:50:07.344856 28837 leader_election.cc:302] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308b15003edc42a48b5f9aecb3fadedb, e711fc080b1c4dfbb6ca86a84722e25b; no voters:
I20260619 18:50:07.345155 29107 raft_consensus.cc:2781] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:50:07.345237 29107 raft_consensus.cc:482] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:50:07.345288 29107 raft_consensus.cc:3037] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:50:07.346380 29107 raft_consensus.cc:504] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:07.346738 29107 leader_election.cc:288] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [CANDIDATE]: Term 1 election: Requested vote from peers 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163), a3c8ff331acc44d8ad159bd0ae36fa0f (127.12.234.193:37799)
I20260619 18:50:07.347276 28768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5209b7922024b48b23315f10049f3f8" candidate_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f"
I20260619 18:50:07.347276 29036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5209b7922024b48b23315f10049f3f8" candidate_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308b15003edc42a48b5f9aecb3fadedb"
I20260619 18:50:07.347429 28768 raft_consensus.cc:3037] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:50:07.347429 29036 raft_consensus.cc:3037] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:50:07.348801 29036 raft_consensus.cc:2454] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e711fc080b1c4dfbb6ca86a84722e25b in term 1.
I20260619 18:50:07.348805 28768 raft_consensus.cc:2454] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e711fc080b1c4dfbb6ca86a84722e25b in term 1.
I20260619 18:50:07.349167 28837 leader_election.cc:302] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308b15003edc42a48b5f9aecb3fadedb, e711fc080b1c4dfbb6ca86a84722e25b; no voters:
I20260619 18:50:07.349349 29107 raft_consensus.cc:2781] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:50:07.349602 29107 raft_consensus.cc:686] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 1 LEADER]: Becoming Leader. State: Replica: e711fc080b1c4dfbb6ca86a84722e25b, State: Running, Role: LEADER
I20260619 18:50:07.349985 29107 consensus_queue.cc:237] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:07.353272 28623 catalog_manager.cc:5495] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b reported cstate change: term changed from 0 to 1, leader changed from <none> to e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194). New cstate: current_term: 1 leader_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20260619 18:50:07.377720 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.374222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40240 (local address 127.12.234.194:36703)
0619 18:50:07.374363 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:07.374368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:07.374382 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:50:07.374467 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:07.374470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:07.374534 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:07.374637 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:50:07.374877 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.375383 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.376254 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:07.376468 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:07.376564 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:07.376877 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:50:07.377323 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:50:07.377441 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:07.377507 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 18:50:07.377574 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 18:50:07.384130 29036 raft_consensus.cc:1261] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [term 1 FOLLOWER]: Refusing update from remote peer e711fc080b1c4dfbb6ca86a84722e25b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:50:07.384171 28768 raft_consensus.cc:1261] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Refusing update from remote peer e711fc080b1c4dfbb6ca86a84722e25b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:50:07.384891 29109 consensus_queue.cc:1035] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [LEADER]: Connected to new peer: Peer: permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:50:07.385099 29107 consensus_queue.cc:1035] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:50:07.393831 29114 mvcc.cc:205] Tried to move back new op lower bound from 7298641950237290496 to 7298641950108434432. Current Snapshot: MvccSnapshot[applied={T|T < 7298641950237290496}]
I20260619 18:50:07.401571 28623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:50:07.401741 28623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:50:07.404287 28623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Sending DeleteTablet for 3 replicas of tablet d5209b7922024b48b23315f10049f3f8
I20260619 18:50:07.405194 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 28685
I20260619 18:50:07.405200 29016 tablet_service.cc:1514] Processing DeleteTablet for tablet d5209b7922024b48b23315f10049f3f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:50:07 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:60736
I20260619 18:50:07.405359 28882 tablet_service.cc:1514] Processing DeleteTablet for tablet d5209b7922024b48b23315f10049f3f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:50:07 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:40230
I20260619 18:50:07.406396 29123 tablet_replica.cc:330] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: stopping tablet replica
I20260619 18:50:07.406831 29123 raft_consensus.cc:2229] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:50:07.407307 29123 raft_consensus.cc:2258] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:50:07.407730 29122 tablet_replica.cc:330] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: stopping tablet replica
I20260619 18:50:07.408089 29122 raft_consensus.cc:2229] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:50:07.408437 29122 raft_consensus.cc:2258] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:50:07.409931 29123 ts_tablet_manager.cc:1905] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 18:50:07.411793 28609 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:37799 (error 108)
I20260619 18:50:07.412024 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 28819
I20260619 18:50:07.413507 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.413187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:37799 (local address 127.0.0.1:54376)
0619 18:50:07.413326 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:07.413405 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:37799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 18:50:07.413849 28609 catalog_manager.cc:4584] TS a3c8ff331acc44d8ad159bd0ae36fa0f (127.12.234.193:37799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5209b7922024b48b23315f10049f3f8: Network error: Client connection negotiation failed: client connection to 127.12.234.193:37799: connect: Connection refused (error 111)
I20260619 18:50:07.414691 29122 ts_tablet_manager.cc:1905] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:50:07.417651 29122 ts_tablet_manager.cc:1918] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:50:07.417778 29122 log.cc:1196] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/d5209b7922024b48b23315f10049f3f8
I20260619 18:50:07.418071 29122 ts_tablet_manager.cc:1939] T d5209b7922024b48b23315f10049f3f8 P 308b15003edc42a48b5f9aecb3fadedb: Deleting consensus metadata
I20260619 18:50:07.419684 28609 catalog_manager.cc:4841] TS 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163): tablet d5209b7922024b48b23315f10049f3f8 (table test-table [id=a4bb7bbdb07d45be9d276b79904aa52b]) successfully deleted
I20260619 18:50:07.419786 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 28953
I20260619 18:50:07.421221 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:07.419999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:36703 (local address 127.0.0.1:40248)
0619 18:50:07.421061 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:07.421123 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.194:36703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":967}
I20260619 18:50:07.424667 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 28591
I20260619 18:50:27.430533 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35779
--webserver_interface=127.12.234.254
--webserver_port=40985
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:35779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:50:27.543406 29125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:27.543708 29125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:27.543751 29125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:27.547930 29125 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 18:50:27.548000 29125 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 18:50:27.548015 29125 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 18:50:27.548028 29125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:50:27.548046 29125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:27.548063 29125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:27.548084 29125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:50:27.548113 29125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:50:27.553732 29125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:35779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=40985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:27.554975 29125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:27.556102 29125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:27.569989 29131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:27.570082 29125 server_base.cc:1034] running on GCE node
W20260619 18:50:27.569993 29132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:27.569993 29134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:27.570915 29125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:27.572247 29125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:27.573501 29125 hybrid_clock.cc:648] HybridClock initialized: now 1781895027573473 us; error 52 us; skew 500 ppm
Jun 19 18:50:27 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895027, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:27.576507 29125 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:50:27.577522 29125 webserver.cc:460] Webserver started at http://127.12.234.254:40985/ using document root <none> and password file <none>
I20260619 18:50:27.578111 29125 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:27.578212 29125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:27.582590 29125 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260619 18:50:27.585129 29141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:27.586599 29125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:50:27.586756 29125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "bd7b152b33344b5b9c93fb7517152aed"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "45dc860dbc2ef8e44c528d656727e58d"
server_key_iv: "135915d9f66ba615d2f0ce506019403b"
server_key_version: "encryptionkey@0"
I20260619 18:50:27.587466 29125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:27.608901 29125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:27.612465 29125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:27.612675 29125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:27.620286 29125 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:35779
I20260619 18:50:27.620572 29193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:35779 every 8 connection(s)
I20260619 18:50:27.621483 29125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:50:27.626264 29194 sys_catalog.cc:263] Verifying existing consensus state
I20260619 18:50:27.627599 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 29125
I20260619 18:50:27.628022 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:37799
--local_ip_for_outbound_sockets=127.12.234.193
--tserver_master_addrs=127.12.234.254:35779
--webserver_port=34469
--webserver_interface=127.12.234.193
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:50:27.628203 29194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Bootstrap starting.
I20260619 18:50:27.646008 29194 log.cc:824] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Log is configured to *not* fsync() on all Append() calls
I20260619 18:50:27.655514 29194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:50:27.655891 29194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Bootstrap complete.
I20260619 18:50:27.659859 29194 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:27.660266 29194 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd7b152b33344b5b9c93fb7517152aed, State: Initialized, Role: FOLLOWER
I20260619 18:50:27.660774 29194 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:27.660902 29194 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:50:27.660969 29194 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:50:27.661065 29194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 1 FOLLOWER]: Advancing to term 2
I20260619 18:50:27.663091 29194 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:27.663416 29194 leader_election.cc:302] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd7b152b33344b5b9c93fb7517152aed; no voters:
I20260619 18:50:27.663722 29194 leader_election.cc:288] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 18:50:27.663821 29198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 2 FOLLOWER]: Leader election won for term 2
I20260619 18:50:27.664029 29198 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 2 LEADER]: Becoming Leader. State: Replica: bd7b152b33344b5b9c93fb7517152aed, State: Running, Role: LEADER
I20260619 18:50:27.664327 29198 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:27.665033 29194 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:50:27.665338 29198 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd7b152b33344b5b9c93fb7517152aed. Latest consensus state: current_term: 2 leader_uuid: "bd7b152b33344b5b9c93fb7517152aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } } }
I20260619 18:50:27.665575 29198 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: This master's current role is: LEADER
I20260619 18:50:27.666200 29199 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bd7b152b33344b5b9c93fb7517152aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } } }
I20260619 18:50:27.666356 29199 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: This master's current role is: LEADER
I20260619 18:50:27.666935 29202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:50:27.671171 29202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:50:27.671726 29202 catalog_manager.cc:1259] Loaded cluster ID: fda11037b1df4de2883ea8009b8646dd
I20260619 18:50:27.671780 29202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:50:27.673593 29202 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:50:27.674271 29202 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Loaded TSK: 0
I20260619 18:50:27.689617 29202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Generated new TSK 1
I20260619 18:50:27.689908 29202 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 18:50:27.758641 29196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:27.758971 29196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:27.759038 29196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:27.763329 29196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:27.763412 29196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:27.763509 29196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:50:27.769696 29196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:37799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=34469
--tserver_master_addrs=127.12.234.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:27.770898 29196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:27.772044 29196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:27.783267 29222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:27.783262 29224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:27.783314 29221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:27.783684 29196 server_base.cc:1034] running on GCE node
I20260619 18:50:27.784103 29196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:27.784775 29196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:27.785946 29196 hybrid_clock.cc:648] HybridClock initialized: now 1781895027785911 us; error 52 us; skew 500 ppm
Jun 19 18:50:27 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895027, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:27.788719 29196 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:50:27.789677 29196 webserver.cc:460] Webserver started at http://127.12.234.193:34469/ using document root <none> and password file <none>
I20260619 18:50:27.790270 29196 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:27.790349 29196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:27.794781 29196 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:50:27.797425 29231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:27.798897 29196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:50:27.799038 29196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "fec2848dcfa7284b4ea570f3069e1043"
server_key_iv: "c4fa37caf2f9644b1e06af857deb1313"
server_key_version: "encryptionkey@0"
I20260619 18:50:27.799671 29196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:27.826011 29196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:27.829675 29196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:27.829885 29196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:27.830602 29196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:50:27.831852 29238 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 18:50:27.836225 29196 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 18:50:27.836333 29196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260619 18:50:27.836421 29196 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 18:50:27.838310 29196 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 18:50:27.838364 29196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260619 18:50:27.838641 29238 tablet_bootstrap.cc:492] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Bootstrap starting.
I20260619 18:50:27.849558 29196 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:37799
I20260619 18:50:27.849699 29345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:37799 every 8 connection(s)
I20260619 18:50:27.850710 29196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:50:27.855204 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 29196
I20260619 18:50:27.855711 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:36703
--local_ip_for_outbound_sockets=127.12.234.194
--tserver_master_addrs=127.12.234.254:35779
--webserver_port=36025
--webserver_interface=127.12.234.194
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:50:27.857303 29238 log.cc:824] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Log is configured to *not* fsync() on all Append() calls
Jun 19 18:50:27 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895027, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:50:27.865885 29238 tablet_bootstrap.cc:492] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:50:27.866369 29238 tablet_bootstrap.cc:492] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Bootstrap complete.
I20260619 18:50:27.867714 29238 ts_tablet_manager.cc:1397] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Time spent bootstrapping tablet: real 0.029s user 0.021s sys 0.007s
I20260619 18:50:27.871089 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:27.853220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:55139 (local address 127.12.234.254:35779)
0619 18:50:27.853701 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:27.853722 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:27.854125 (+ 403us) server_negotiation.cc:407] Connection header received
0619 18:50:27.855732 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:27.855750 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:27.856244 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:27.856646 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:27.858062 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:27.859546 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:27.861213 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:27.861967 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:27.862160 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:27.865233 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:27.865240 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:27.865257 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:27.865292 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:27.867409 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:27.868061 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:27.868068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:27.868074 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:27.868239 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:27.868628 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:27.868631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:27.868633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:27.869102 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:27.869235 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:27.869871 (+ 636us) server_negotiation.cc:299] Negotiation successful
0619 18:50:27.870215 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":155,"threads_started":1}
I20260619 18:50:27.871105 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:27.853202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:55139)
0619 18:50:27.853918 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:27.853981 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:27.855195 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:27.856735 (+ 1540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:27.856744 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:27.857241 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:50:27.857880 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:27.857897 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:27.859732 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:27.859735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:27.861080 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:27.861091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:27.862137 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:27.862141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:27.862354 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:27.863034 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:50:27.863054 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:50:27.864999 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:50:27.867536 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:27.867543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:27.867558 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:27.867902 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:27.868344 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:27.868347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:27.868349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:27.868517 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:27.869210 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:50:27.869216 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:50:27.869531 (+ 315us) client_negotiation.cc:769] Sending connection context
0619 18:50:27.869811 (+ 280us) client_negotiation.cc:240] Negotiation successful
0619 18:50:27.870155 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":162,"threads_started":1}
I20260619 18:50:27.870725 29238 raft_consensus.cc:348] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:27.871240 29238 raft_consensus.cc:729] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c8ff331acc44d8ad159bd0ae36fa0f, State: Initialized, Role: FOLLOWER
I20260619 18:50:27.871860 29238 consensus_queue.cc:260] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:27.872810 29238 ts_tablet_manager.cc:1428] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260619 18:50:27.873803 29346 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35779
I20260619 18:50:27.874032 29346 heartbeater.cc:464] Registering TS with master...
I20260619 18:50:27.874539 29346 heartbeater.cc:511] Master 127.12.234.254:35779 requested a full tablet report, sending...
I20260619 18:50:27.876861 29158 ts_manager.cc:194] Registered new tserver with Master: a3c8ff331acc44d8ad159bd0ae36fa0f (127.12.234.193:37799)
I20260619 18:50:27.880616 29158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:55139
I20260619 18:50:27.883010 29346 heartbeater.cc:503] Master 127.12.234.254:35779 was elected leader, sending a full tablet report...
Jun 19 18:50:27 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895027, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.193@KRBTEST.COM
I20260619 18:50:27.890715 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:27.879760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:37799 (local address 127.0.0.1:54322)
0619 18:50:27.880208 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:27.880255 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:27.880383 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:27.881092 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:27.881098 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:27.881570 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:50:27.881705 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:27.881714 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:27.883208 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:27.883212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:27.884143 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:27.884154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:27.884776 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:27.884780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:27.884886 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:27.885430 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:50:27.885444 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:50:27.887086 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:50:27.888903 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:27.888912 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:27.888918 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:27.889145 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:27.889420 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:27.889423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:27.889425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:27.889525 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:27.889955 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:50:27.889961 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:50:27.890099 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 18:50:27.890282 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 18:50:27.890504 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":137,"threads_started":1}
I20260619 18:50:27.890720 29354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:27.879843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.12.234.193:37799)
0619 18:50:27.880585 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:27.880596 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:27.880619 (+ 23us) server_negotiation.cc:407] Connection header received
0619 18:50:27.880689 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:27.880740 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:27.880940 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:27.881083 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:27.881830 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:27.882949 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:27.884268 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:27.884667 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:27.884752 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:27.887234 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:27.887246 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:27.887255 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:27.887277 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:27.888796 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:27.889251 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:27.889257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:27.889258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:27.889335 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:27.889606 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:27.889609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:27.889611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:27.889848 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:27.889970 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:27.890364 (+ 394us) server_negotiation.cc:299] Negotiation successful
0619 18:50:27.890537 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":108,"threads_started":1}
I20260619 18:50:27.892189 29280 tablet_service.cc:1514] Processing DeleteTablet for tablet d5209b7922024b48b23315f10049f3f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:50:07 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:54322
I20260619 18:50:27.893258 29355 tablet_replica.cc:330] stopping tablet replica
I20260619 18:50:27.893580 29355 raft_consensus.cc:2229] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:50:27.893821 29355 raft_consensus.cc:2258] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:50:27.894853 29355 ts_tablet_manager.cc:1905] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:50:27.898804 29355 ts_tablet_manager.cc:1918] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:50:27.898912 29355 log.cc:1196] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal/wals/d5209b7922024b48b23315f10049f3f8
I20260619 18:50:27.899127 29355 ts_tablet_manager.cc:1939] T d5209b7922024b48b23315f10049f3f8 P a3c8ff331acc44d8ad159bd0ae36fa0f: Deleting consensus metadata
I20260619 18:50:27.901101 29144 catalog_manager.cc:4841] TS a3c8ff331acc44d8ad159bd0ae36fa0f (127.12.234.193:37799): tablet d5209b7922024b48b23315f10049f3f8 (table test-table [id=a4bb7bbdb07d45be9d276b79904aa52b]) successfully deleted
W20260619 18:50:27.977146 29350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:27.977427 29350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:27.977471 29350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:27.981793 29350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:27.981863 29350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:27.981925 29350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:50:27.987351 29350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:36703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=36025
--tserver_master_addrs=127.12.234.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:27.988400 29350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:27.989441 29350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:27.999682 29361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:27.999682 29360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:27.999819 29350 server_base.cc:1034] running on GCE node
W20260619 18:50:27.999682 29363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:28.000387 29350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:28.001052 29350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:28.002265 29350 hybrid_clock.cc:648] HybridClock initialized: now 1781895028002238 us; error 38 us; skew 500 ppm
Jun 19 18:50:28 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:28.005365 29350 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:50:28.006392 29350 webserver.cc:460] Webserver started at http://127.12.234.194:36025/ using document root <none> and password file <none>
I20260619 18:50:28.006997 29350 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:28.007083 29350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:28.011194 29350 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:50:28.013672 29370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:28.015272 29350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 18:50:28.015399 29350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "e711fc080b1c4dfbb6ca86a84722e25b"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "aee7442d1b3cba728d4fe76ff0c0b20f"
server_key_iv: "5742c407a70ba73d0a2fe35f3cb6cbf5"
server_key_version: "encryptionkey@0"
I20260619 18:50:28.016004 29350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:28.036202 29350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:28.040241 29350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:28.040477 29350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:28.041152 29350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:50:28.042414 29377 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260619 18:50:28.046538 29377 ts_tablet_manager.cc:1852] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260619 18:50:28.046972 29377 ts_tablet_manager.cc:1905] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:50:28.051331 29377 ts_tablet_manager.cc:1918] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:50:28.051452 29377 log.cc:1196] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/d5209b7922024b48b23315f10049f3f8
I20260619 18:50:28.051774 29377 ts_tablet_manager.cc:1939] T d5209b7922024b48b23315f10049f3f8 P e711fc080b1c4dfbb6ca86a84722e25b: Deleting consensus metadata
I20260619 18:50:28.052460 29350 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260619 18:50:28.052532 29350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.001s
I20260619 18:50:28.052605 29350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:50:28.052631 29350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:28.061892 29350 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:36703
I20260619 18:50:28.062021 29484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:36703 every 8 connection(s)
I20260619 18:50:28.063017 29350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:50:28.063325 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 29350
I20260619 18:50:28.063798 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:33163
--local_ip_for_outbound_sockets=127.12.234.195
--tserver_master_addrs=127.12.234.254:35779
--webserver_port=37461
--webserver_interface=127.12.234.195
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 18:50:28 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:50:28.080439 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.065278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:49841 (local address 127.12.234.254:35779)
0619 18:50:28.065450 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.065455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.066687 (+ 1232us) server_negotiation.cc:407] Connection header received
0619 18:50:28.068049 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.068053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.068115 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.068193 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:28.070071 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.070699 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.072537 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.072795 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.072906 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.076226 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:28.076230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:28.076232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:28.076264 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:28.077785 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:28.078490 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:28.078494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:28.078496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:28.078594 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:28.079052 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:28.079056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:28.079057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:28.079258 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:28.079334 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:28.080116 (+ 782us) server_negotiation.cc:299] Negotiation successful
0619 18:50:28.080266 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 18:50:28.081565 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.065785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:49841)
0619 18:50:28.066416 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:28.066560 (+ 144us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:28.067731 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:28.068455 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:28.068466 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:28.069019 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:50:28.069829 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.069853 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.070854 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.070858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.072372 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.072387 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.072989 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.072995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.073225 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.073965 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:50:28.073989 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:50:28.075985 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:50:28.077928 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:28.077935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:28.077956 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:28.078372 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:28.078685 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:28.078688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:28.078690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:28.078890 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:28.079346 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:50:28.079353 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:50:28.079723 (+ 370us) client_negotiation.cc:769] Sending connection context
0619 18:50:28.080036 (+ 313us) client_negotiation.cc:240] Negotiation successful
0619 18:50:28.080430 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":142,"threads_started":1}
I20260619 18:50:28.082994 29485 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35779
I20260619 18:50:28.083266 29485 heartbeater.cc:464] Registering TS with master...
I20260619 18:50:28.083822 29485 heartbeater.cc:511] Master 127.12.234.254:35779 requested a full tablet report, sending...
I20260619 18:50:28.085098 29158 ts_manager.cc:194] Registered new tserver with Master: e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703)
I20260619 18:50:28.085727 29158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:49841
W20260619 18:50:28.183223 29487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:28.183530 29487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:28.183593 29487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:28.187829 29487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:28.187908 29487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:28.187994 29487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:50:28.193959 29487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:33163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=37461
--tserver_master_addrs=127.12.234.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:28.195129 29487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:28.196348 29487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:28.206346 29494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:28.206368 29487 server_base.cc:1034] running on GCE node
W20260619 18:50:28.206324 29493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:28.206317 29496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:28.207018 29487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:28.207692 29487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:28.208856 29487 hybrid_clock.cc:648] HybridClock initialized: now 1781895028208840 us; error 28 us; skew 500 ppm
Jun 19 18:50:28 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:28.211630 29487 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:50:28.212569 29487 webserver.cc:460] Webserver started at http://127.12.234.195:37461/ using document root <none> and password file <none>
I20260619 18:50:28.213119 29487 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:28.213196 29487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:28.217123 29487 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:50:28.219498 29503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:28.220980 29487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:50:28.221092 29487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "308b15003edc42a48b5f9aecb3fadedb"
format_stamp: "Formatted at 2026-06-19 18:50:07 on dist-test-slave-fljv"
server_key: "30e31d8db5b011ffc166eaf9bf2dfe45"
server_key_iv: "2f229c4e03d1e8bf1f0431b2846cc403"
server_key_version: "encryptionkey@0"
I20260619 18:50:28.221655 29487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:28.251938 29487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:28.255225 29487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:28.255414 29487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:28.256064 29487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:50:28.256927 29487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:50:28.256973 29487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:28.257014 29487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:50:28.257043 29487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:28.265925 29487 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:33163
I20260619 18:50:28.265990 29616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:33163 every 8 connection(s)
I20260619 18:50:28.267050 29487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:50:28.271675 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 29487
Jun 19 18:50:28 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:50:28.280464 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.272480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.12.234.254:35779)
0619 18:50:28.272798 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.272802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.272818 (+ 16us) server_negotiation.cc:407] Connection header received
0619 18:50:28.272922 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.272925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.272969 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.273063 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:28.273852 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.274517 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.275372 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.275717 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.275801 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.277182 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:28.277186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:28.277189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:28.277216 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:28.278777 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:28.279180 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:28.279184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:28.279186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:28.279259 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:28.279509 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:28.279513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:28.279514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:28.279688 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:28.279884 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:28.280041 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 18:50:28.280159 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":245,"spinlock_wait_cycles":9984,"thread_start_us":90,"threads_started":1}
I20260619 18:50:28.281848 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.269189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:60917 (local address 127.12.234.254:35779)
0619 18:50:28.269348 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.269353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.270552 (+ 1199us) server_negotiation.cc:407] Connection header received
0619 18:50:28.271487 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.271491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.271553 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.271661 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:28.273417 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.274143 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.275419 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.275721 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.275810 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.278521 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:28.278524 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:28.278527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:28.278549 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:28.279775 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:28.280252 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:28.280255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:28.280257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:28.280324 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:28.280662 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:28.280665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:28.280667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:28.280844 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:28.280916 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:28.281569 (+ 653us) server_negotiation.cc:299] Negotiation successful
0619 18:50:28.281698 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":58,"spinlock_wait_cycles":6912}
I20260619 18:50:28.282814 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.269641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:60917)
0619 18:50:28.270382 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:28.270429 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:28.271253 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:28.271897 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:28.271905 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:28.272566 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:50:28.273266 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.273289 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.274282 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.274286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.275298 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.275308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.275870 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.275874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.276050 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.276611 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:50:28.276627 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:50:28.278341 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:50:28.279883 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:28.279890 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:28.279907 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:28.280164 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:28.280403 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:28.280406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:28.280408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:28.280568 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:28.280929 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:50:28.280936 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:50:28.281279 (+ 343us) client_negotiation.cc:769] Sending connection context
0619 18:50:28.281495 (+ 216us) client_negotiation.cc:240] Negotiation successful
0619 18:50:28.281806 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":115,"threads_started":1}
I20260619 18:50:28.284001 29617 heartbeater.cc:347] Connected to a master server at 127.12.234.254:35779
I20260619 18:50:28.284237 29617 heartbeater.cc:464] Registering TS with master...
I20260619 18:50:28.284721 29617 heartbeater.cc:511] Master 127.12.234.254:35779 requested a full tablet report, sending...
I20260619 18:50:28.285689 29158 ts_manager.cc:194] Registered new tserver with Master: 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163)
I20260619 18:50:28.286284 29158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:60917
I20260619 18:50:28.294559 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:50:28.299286 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.295376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.12.234.254:35779)
0619 18:50:28.295538 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.295541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.295687 (+ 146us) server_negotiation.cc:407] Connection header received
0619 18:50:28.295810 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.295812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.295931 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.296078 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:50:28.296338 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.296968 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.297992 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.298264 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.298357 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.298714 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:50:28.298865 (+ 151us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 18:50:28.299156 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52294: token expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260619 18:50:28.299381 29348 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52294: token expired
W20260619 18:50:28.299752 29086 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.12.234.254:35779)
I20260619 18:50:28.307202 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.300111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52310 (local address 127.12.234.254:35779)
0619 18:50:28.300234 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.300238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.300250 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:50:28.300345 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.300348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.300393 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.300462 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:28.301078 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.301604 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.302454 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.302670 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.302756 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.304089 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:28.304092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:28.304094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:28.304118 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:28.305539 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:28.306033 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:28.306037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:28.306038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:28.306146 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:28.306464 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:28.306467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:28.306468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:28.306661 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:28.306771 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:28.306981 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 18:50:28.307078 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 18:50:28.310009 29158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:50:28.313580 29158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:50:28.321781 29280 tablet_service.cc:1467] Processing CreateTablet for tablet 681e085b4a4c4728aba6560447be36ab (DEFAULT_TABLE table=test-table [id=1211c6218980475b801b211f95d6149d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:50:28.322482 29280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 681e085b4a4c4728aba6560447be36ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:50:28.325507 29238 tablet_bootstrap.cc:492] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Bootstrap starting.
I20260619 18:50:28.326467 29238 tablet_bootstrap.cc:654] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Neither blocks nor log segments found. Creating new log.
I20260619 18:50:28.326697 29626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.320886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:33163 (local address 127.0.0.1:39888)
0619 18:50:28.321465 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:28.321478 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:28.321572 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:28.322043 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:28.322046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:28.322064 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:50:28.322208 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.322215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.323666 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.323670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.324970 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.324981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.326235 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.326239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.326336 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.326404 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 18:50:28.326457 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 18:50:28.326542 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":1,"thread_start_us":121,"threads_started":1}
I20260619 18:50:28.327293 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.320622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:36703 (local address 127.0.0.1:42254)
0619 18:50:28.320734 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:28.320749 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:28.320829 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:28.323203 (+ 2374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:28.323206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:28.323223 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:50:28.323315 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.323321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.324509 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.324512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.325648 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.325659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.326930 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.326934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.327032 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.327052 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 18:50:28.327102 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 18:50:28.327176 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":52}
I20260619 18:50:28.327358 29627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.321087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39888 (local address 127.12.234.195:33163)
0619 18:50:28.321478 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.321492 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.321558 (+ 66us) server_negotiation.cc:407] Connection header received
0619 18:50:28.321694 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.321705 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.321907 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.322059 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:50:28.322311 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.323551 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.325092 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.326079 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.326919 (+ 840us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.327076 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:28.327130 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 18:50:28.327229 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":157,"threads_started":1}
I20260619 18:50:28.327998 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.322357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42254 (local address 127.12.234.194:36703)
0619 18:50:28.322729 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.322740 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.322763 (+ 23us) server_negotiation.cc:407] Connection header received
0619 18:50:28.322839 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.322845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.323050 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.323465 (+ 415us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:50:28.323625 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.324409 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.325825 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.326704 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.327586 (+ 882us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.327728 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:28.327778 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 18:50:28.327869 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":154,"threads_started":1}
I20260619 18:50:28.329102 29238 tablet_bootstrap.cc:492] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: No bootstrap required, opened a new log
I20260619 18:50:28.329205 29238 ts_tablet_manager.cc:1397] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 18:50:28.329788 29238 raft_consensus.cc:348] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:28.329928 29238 raft_consensus.cc:374] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:50:28.329970 29238 raft_consensus.cc:729] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c8ff331acc44d8ad159bd0ae36fa0f, State: Initialized, Role: FOLLOWER
I20260619 18:50:28.330093 29238 consensus_queue.cc:260] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:28.329653 29419 tablet_service.cc:1467] Processing CreateTablet for tablet 681e085b4a4c4728aba6560447be36ab (DEFAULT_TABLE table=test-table [id=1211c6218980475b801b211f95d6149d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:50:28.330479 29238 ts_tablet_manager.cc:1428] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 18:50:28.330854 29419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 681e085b4a4c4728aba6560447be36ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:50:28.330574 29551 tablet_service.cc:1467] Processing CreateTablet for tablet 681e085b4a4c4728aba6560447be36ab (DEFAULT_TABLE table=test-table [id=1211c6218980475b801b211f95d6149d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:50:28.331473 29551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 681e085b4a4c4728aba6560447be36ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:50:28.335516 29377 tablet_bootstrap.cc:492] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: Bootstrap starting.
I20260619 18:50:28.337572 29377 tablet_bootstrap.cc:654] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: Neither blocks nor log segments found. Creating new log.
I20260619 18:50:28.338385 29630 tablet_bootstrap.cc:492] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: Bootstrap starting.
I20260619 18:50:28.338414 29377 log.cc:824] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: Log is configured to *not* fsync() on all Append() calls
I20260619 18:50:28.340896 29377 tablet_bootstrap.cc:492] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: No bootstrap required, opened a new log
I20260619 18:50:28.341092 29377 ts_tablet_manager.cc:1397] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260619 18:50:28.341195 29630 tablet_bootstrap.cc:654] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: Neither blocks nor log segments found. Creating new log.
I20260619 18:50:28.341924 29630 log.cc:824] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: Log is configured to *not* fsync() on all Append() calls
I20260619 18:50:28.344094 29630 tablet_bootstrap.cc:492] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: No bootstrap required, opened a new log
I20260619 18:50:28.344319 29630 ts_tablet_manager.cc:1397] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260619 18:50:28.344666 29377 raft_consensus.cc:348] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:28.344935 29377 raft_consensus.cc:374] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:50:28.344995 29377 raft_consensus.cc:729] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e711fc080b1c4dfbb6ca86a84722e25b, State: Initialized, Role: FOLLOWER
I20260619 18:50:28.345420 29377 consensus_queue.cc:260] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:28.346120 29485 heartbeater.cc:503] Master 127.12.234.254:35779 was elected leader, sending a full tablet report...
I20260619 18:50:28.346462 29377 ts_tablet_manager.cc:1428] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: Time spent starting tablet: real 0.005s user 0.002s sys 0.005s
I20260619 18:50:28.347762 29630 raft_consensus.cc:348] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:28.348114 29630 raft_consensus.cc:374] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:50:28.348176 29630 raft_consensus.cc:729] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308b15003edc42a48b5f9aecb3fadedb, State: Initialized, Role: FOLLOWER
I20260619 18:50:28.348703 29630 consensus_queue.cc:260] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:28.349481 29617 heartbeater.cc:503] Master 127.12.234.254:35779 was elected leader, sending a full tablet report...
I20260619 18:50:28.349876 29630 ts_tablet_manager.cc:1428] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260619 18:50:28.351941 29347 tablet.cc:2369] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:50:28.486321 29634 raft_consensus.cc:482] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:50:28.486588 29634 raft_consensus.cc:504] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:28.487768 29634 leader_election.cc:288] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703), a3c8ff331acc44d8ad159bd0ae36fa0f (127.12.234.193:37799)
I20260619 18:50:28.491418 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.488195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:56257 (local address 127.12.234.194:36703)
0619 18:50:28.488346 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.488351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.488376 (+ 25us) server_negotiation.cc:407] Connection header received
0619 18:50:28.488435 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.488437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.488491 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.488586 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:50:28.488859 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.489355 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.490428 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.490884 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.490990 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.491046 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:28.491236 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 18:50:28.491303 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 18:50:28.491418 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.488049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:36703 (local address 127.12.234.195:56257)
0619 18:50:28.488201 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:28.488218 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:28.488338 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:28.488615 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:28.488619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:28.488637 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:50:28.488747 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.488753 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.489472 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.489475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.490311 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.490320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.490991 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.490994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.491121 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.491141 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 18:50:28.491204 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 18:50:28.491277 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":1}
I20260619 18:50:28.492058 29635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.488320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:37799 (local address 127.12.234.195:47923)
0619 18:50:28.488627 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:28.488640 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:28.488726 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:28.489044 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:28.489046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:28.489062 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:50:28.489165 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.489169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.489905 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.489909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.490777 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:28.490786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.491680 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.491684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:28.491772 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.491794 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:50:28.491860 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 18:50:28.491923 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":24,"thread_start_us":112,"threads_started":1}
I20260619 18:50:28.492089 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.488473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:47923 (local address 127.12.234.193:37799)
0619 18:50:28.488769 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.488773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.488788 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:50:28.488839 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.488841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.488890 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.488988 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:50:28.489272 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.489796 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.490890 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.491418 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.491535 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.491704 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:28.491898 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 18:50:28.491976 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":106,"threads_started":1}
I20260619 18:50:28.492096 29439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "681e085b4a4c4728aba6560447be36ab" candidate_uuid: "308b15003edc42a48b5f9aecb3fadedb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" is_pre_election: true
I20260619 18:50:28.492465 29439 raft_consensus.cc:2454] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308b15003edc42a48b5f9aecb3fadedb in term 0.
I20260619 18:50:28.492635 29300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "681e085b4a4c4728aba6560447be36ab" candidate_uuid: "308b15003edc42a48b5f9aecb3fadedb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" is_pre_election: true
I20260619 18:50:28.492933 29300 raft_consensus.cc:2454] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308b15003edc42a48b5f9aecb3fadedb in term 0.
I20260619 18:50:28.492961 29506 leader_election.cc:302] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308b15003edc42a48b5f9aecb3fadedb, e711fc080b1c4dfbb6ca86a84722e25b; no voters:
I20260619 18:50:28.493224 29634 raft_consensus.cc:2781] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:50:28.493309 29634 raft_consensus.cc:482] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:50:28.493356 29634 raft_consensus.cc:3037] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:50:28.494436 29634 raft_consensus.cc:504] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:28.494755 29634 leader_election.cc:288] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [CANDIDATE]: Term 1 election: Requested vote from peers e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703), a3c8ff331acc44d8ad159bd0ae36fa0f (127.12.234.193:37799)
I20260619 18:50:28.495100 29439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "681e085b4a4c4728aba6560447be36ab" candidate_uuid: "308b15003edc42a48b5f9aecb3fadedb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e711fc080b1c4dfbb6ca86a84722e25b"
I20260619 18:50:28.495209 29300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "681e085b4a4c4728aba6560447be36ab" candidate_uuid: "308b15003edc42a48b5f9aecb3fadedb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f"
I20260619 18:50:28.495230 29439 raft_consensus.cc:3037] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:50:28.495352 29300 raft_consensus.cc:3037] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:50:28.496342 29439 raft_consensus.cc:2454] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308b15003edc42a48b5f9aecb3fadedb in term 1.
I20260619 18:50:28.496403 29300 raft_consensus.cc:2454] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308b15003edc42a48b5f9aecb3fadedb in term 1.
I20260619 18:50:28.496659 29506 leader_election.cc:302] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308b15003edc42a48b5f9aecb3fadedb, e711fc080b1c4dfbb6ca86a84722e25b; no voters:
I20260619 18:50:28.496860 29634 raft_consensus.cc:2781] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:50:28.497079 29634 raft_consensus.cc:686] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 1 LEADER]: Becoming Leader. State: Replica: 308b15003edc42a48b5f9aecb3fadedb, State: Running, Role: LEADER
I20260619 18:50:28.497401 29634 consensus_queue.cc:237] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:28.500975 29157 catalog_manager.cc:5495] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb reported cstate change: term changed from 0 to 1, leader changed from <none> to 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195). New cstate: current_term: 1 leader_uuid: "308b15003edc42a48b5f9aecb3fadedb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } health_report { overall_health: UNKNOWN } } }
W20260619 18:50:28.518043 29618 tablet.cc:2369] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:50:28.527036 29627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.523004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39898 (local address 127.12.234.195:33163)
0619 18:50:28.523125 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:28.523130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:28.523146 (+ 16us) server_negotiation.cc:407] Connection header received
0619 18:50:28.523278 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:28.523282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:28.523351 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:28.523459 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:50:28.523712 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.524353 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.525349 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:28.525684 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:28.525792 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:28.526162 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:50:28.526676 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:50:28.526774 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:28.526868 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 18:50:28.526921 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:50:28.533082 29300 raft_consensus.cc:1261] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Refusing update from remote peer 308b15003edc42a48b5f9aecb3fadedb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:50:28.533087 29439 raft_consensus.cc:1261] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [term 1 FOLLOWER]: Refusing update from remote peer 308b15003edc42a48b5f9aecb3fadedb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:50:28.533944 29637 consensus_queue.cc:1035] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:50:28.534137 29634 consensus_queue.cc:1035] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:50:28.540839 29640 mvcc.cc:205] Tried to move back new op lower bound from 7298642036863791104 to 7298642036728279040. Current Snapshot: MvccSnapshot[applied={T|T < 7298642036863791104}]
I20260619 18:50:28.541042 29641 mvcc.cc:205] Tried to move back new op lower bound from 7298642036863791104 to 7298642036728279040. Current Snapshot: MvccSnapshot[applied={T|T < 7298642036863791104}]
I20260619 18:50:28.549456 29157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:50:28.549631 29157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:50:28.551985 29157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Sending DeleteTablet for 3 replicas of tablet 681e085b4a4c4728aba6560447be36ab
I20260619 18:50:28.552811 29280 tablet_service.cc:1514] Processing DeleteTablet for tablet 681e085b4a4c4728aba6560447be36ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:50:28 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:54322
I20260619 18:50:28.552904 29419 tablet_service.cc:1514] Processing DeleteTablet for tablet 681e085b4a4c4728aba6560447be36ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:50:28 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:42254
I20260619 18:50:28.552927 29551 tablet_service.cc:1514] Processing DeleteTablet for tablet 681e085b4a4c4728aba6560447be36ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:50:28 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:39888
I20260619 18:50:28.553176 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 29196
I20260619 18:50:28.553889 29651 tablet_replica.cc:330] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: stopping tablet replica
I20260619 18:50:28.554316 29651 raft_consensus.cc:2229] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:50:28.554651 29651 raft_consensus.cc:2258] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:50:28.555390 29652 tablet_replica.cc:330] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: stopping tablet replica
I20260619 18:50:28.555701 29652 raft_consensus.cc:2229] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:50:28.556116 29652 raft_consensus.cc:2258] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:50:28.556305 29651 ts_tablet_manager.cc:1905] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:50:28.557543 29652 ts_tablet_manager.cc:1905] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:50:28.559192 29651 ts_tablet_manager.cc:1918] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:50:28.559348 29651 log.cc:1196] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/681e085b4a4c4728aba6560447be36ab
I20260619 18:50:28.559675 29651 ts_tablet_manager.cc:1939] T 681e085b4a4c4728aba6560447be36ab P e711fc080b1c4dfbb6ca86a84722e25b: Deleting consensus metadata
I20260619 18:50:28.560179 29652 ts_tablet_manager.cc:1918] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:50:28.560276 29652 log.cc:1196] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/681e085b4a4c4728aba6560447be36ab
I20260619 18:50:28.560587 29652 ts_tablet_manager.cc:1939] T 681e085b4a4c4728aba6560447be36ab P 308b15003edc42a48b5f9aecb3fadedb: Deleting consensus metadata
W20260619 18:50:28.561072 29144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:37799 (error 108)
I20260619 18:50:28.561596 29144 catalog_manager.cc:4841] TS 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163): tablet 681e085b4a4c4728aba6560447be36ab (table test-table [id=1211c6218980475b801b211f95d6149d]) successfully deleted
I20260619 18:50:28.561754 29144 catalog_manager.cc:4841] TS e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703): tablet 681e085b4a4c4728aba6560447be36ab (table test-table [id=1211c6218980475b801b211f95d6149d]) successfully deleted
I20260619 18:50:28.562284 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 29350
I20260619 18:50:28.562850 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:28.562575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:37799 (local address 127.0.0.1:54330)
0619 18:50:28.562689 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:28.562764 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:37799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 18:50:28.563030 29144 catalog_manager.cc:4584] TS a3c8ff331acc44d8ad159bd0ae36fa0f (127.12.234.193:37799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 681e085b4a4c4728aba6560447be36ab: Network error: Client connection negotiation failed: client connection to 127.12.234.193:37799: connect: Connection refused (error 111)
I20260619 18:50:28.568967 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 29487
I20260619 18:50:28.574266 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 29125
I20260619 18:50:48.580189 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35779
--webserver_interface=127.12.234.254
--webserver_port=40985
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:35779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:50:48.689046 29654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:48.689333 29654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:48.689376 29654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:48.693441 29654 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 18:50:48.693503 29654 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 18:50:48.693517 29654 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 18:50:48.693531 29654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:50:48.693547 29654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:48.693564 29654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:48.693581 29654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:50:48.693606 29654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:50:48.699015 29654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:35779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:35779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=40985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:48.700111 29654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:48.701154 29654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:48.709971 29659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:48.709951 29654 server_base.cc:1034] running on GCE node
W20260619 18:50:48.709954 29660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:48.709944 29662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:48.710892 29654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:48.712026 29654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:48.713203 29654 hybrid_clock.cc:648] HybridClock initialized: now 1781895048713183 us; error 39 us; skew 500 ppm
Jun 19 18:50:48 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895048, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:48.716132 29654 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:50:48.717115 29654 webserver.cc:460] Webserver started at http://127.12.234.254:40985/ using document root <none> and password file <none>
I20260619 18:50:48.717685 29654 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:48.717749 29654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:48.721786 29654 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:50:48.724282 29669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:48.725760 29654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 18:50:48.725905 29654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "bd7b152b33344b5b9c93fb7517152aed"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "45dc860dbc2ef8e44c528d656727e58d"
server_key_iv: "135915d9f66ba615d2f0ce506019403b"
server_key_version: "encryptionkey@0"
I20260619 18:50:48.726557 29654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:48.750398 29654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:48.753397 29654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:48.753588 29654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:48.760499 29654 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:35779
I20260619 18:50:48.760763 29721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:35779 every 8 connection(s)
I20260619 18:50:48.761590 29654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:50:48.766052 29722 sys_catalog.cc:263] Verifying existing consensus state
I20260619 18:50:48.766896 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 29654
I20260619 18:50:48.767393 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:37799
--local_ip_for_outbound_sockets=127.12.234.193
--tserver_master_addrs=127.12.234.254:35779
--webserver_port=34469
--webserver_interface=127.12.234.193
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:50:48.768239 29722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Bootstrap starting.
I20260619 18:50:48.785977 29722 log.cc:824] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Log is configured to *not* fsync() on all Append() calls
I20260619 18:50:48.797659 29722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:50:48.798069 29722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Bootstrap complete.
I20260619 18:50:48.802083 29722 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:48.802556 29722 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd7b152b33344b5b9c93fb7517152aed, State: Initialized, Role: FOLLOWER
I20260619 18:50:48.803066 29722 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:48.803206 29722 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:50:48.803274 29722 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:50:48.803364 29722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 2 FOLLOWER]: Advancing to term 3
I20260619 18:50:48.806131 29722 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:48.806561 29722 leader_election.cc:302] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd7b152b33344b5b9c93fb7517152aed; no voters:
I20260619 18:50:48.806931 29722 leader_election.cc:288] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 18:50:48.807039 29726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 3 FOLLOWER]: Leader election won for term 3
I20260619 18:50:48.807224 29726 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [term 3 LEADER]: Becoming Leader. State: Replica: bd7b152b33344b5b9c93fb7517152aed, State: Running, Role: LEADER
I20260619 18:50:48.807540 29726 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } }
I20260619 18:50:48.808254 29722 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:50:48.808544 29726 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd7b152b33344b5b9c93fb7517152aed. Latest consensus state: current_term: 3 leader_uuid: "bd7b152b33344b5b9c93fb7517152aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } } }
I20260619 18:50:48.808676 29726 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: This master's current role is: LEADER
I20260619 18:50:48.809254 29727 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bd7b152b33344b5b9c93fb7517152aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b152b33344b5b9c93fb7517152aed" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 35779 } } }
I20260619 18:50:48.809352 29727 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed [sys.catalog]: This master's current role is: LEADER
I20260619 18:50:48.810021 29730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:50:48.815274 29730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:50:48.815961 29730 catalog_manager.cc:1259] Loaded cluster ID: fda11037b1df4de2883ea8009b8646dd
I20260619 18:50:48.816028 29730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:50:48.818580 29730 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:50:48.819330 29730 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Loaded TSK: 0
I20260619 18:50:48.819418 29730 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd7b152b33344b5b9c93fb7517152aed: Loaded TSK: 1
F20260619 18:50:48.820509 29730 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3f3d9f79ad google::LogMessage::Fail() at ??:0
@ 0x7f3f3d9fa88a google::LogMessage::SendToLog() at ??:0
@ 0x7f3f3d9f74de google::LogMessage::Flush() at ??:0
@ 0x7f3f3d9f8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3f406bb18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3f406b3f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3f4835c7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3f4833699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3f48337178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f3f48378df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f3f4839782b std::function<>::operator()() at ??:0
@ 0x7f3f48336b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f3f48337f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3f48334579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f3f48378640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f3f3fc493d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3f3fc49c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f3f3fc4b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f3f3fc3a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3f4127e6db start_thread at ??:0
@ 0x7f3f3c4f561f clone at ??:0
W20260619 18:50:48.913213 29724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:48.913482 29724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:48.913596 29724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:48.917879 29724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:48.917950 29724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:48.918036 29724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:50:48.923553 29724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:37799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=34469
--tserver_master_addrs=127.12.234.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:48.924676 29724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:48.925726 29724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:48.935482 29769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:48.935510 29770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:48.935510 29772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:48.936070 29724 server_base.cc:1034] running on GCE node
I20260619 18:50:48.936477 29724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:48.937150 29724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:48.938306 29724 hybrid_clock.cc:648] HybridClock initialized: now 1781895048938260 us; error 57 us; skew 500 ppm
Jun 19 18:50:48 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895048, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:48.941205 29724 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:50:48.942154 29724 webserver.cc:460] Webserver started at http://127.12.234.193:34469/ using document root <none> and password file <none>
I20260619 18:50:48.942740 29724 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:48.942816 29724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:48.946717 29724 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260619 18:50:48.948979 29779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:48.950421 29724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:50:48.950534 29724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "fec2848dcfa7284b4ea570f3069e1043"
server_key_iv: "c4fa37caf2f9644b1e06af857deb1313"
server_key_version: "encryptionkey@0"
I20260619 18:50:48.951092 29724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:48.968524 29724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:48.972086 29724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:48.972280 29724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:48.972914 29724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:50:48.974085 29786 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 18:50:48.979815 29724 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 18:50:48.979911 29724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260619 18:50:48.979995 29724 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 18:50:48.981863 29724 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 18:50:48.981920 29724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 18:50:48.982295 29786 tablet_bootstrap.cc:492] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Bootstrap starting.
I20260619 18:50:48.991683 29724 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:37799
I20260619 18:50:48.991766 29893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:37799 every 8 connection(s)
I20260619 18:50:48.992657 29724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:50:48.994995 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 29724
I20260619 18:50:48.995482 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:36703
--local_ip_for_outbound_sockets=127.12.234.194
--tserver_master_addrs=127.12.234.254:35779
--webserver_port=36025
--webserver_interface=127.12.234.194
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:50:49.004315 29786 log.cc:824] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Log is configured to *not* fsync() on all Append() calls
I20260619 18:50:49.014320 29786 tablet_bootstrap.cc:492] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 18:50:49.014786 29786 tablet_bootstrap.cc:492] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Bootstrap complete.
I20260619 18:50:49.016193 29786 ts_tablet_manager.cc:1397] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Time spent bootstrapping tablet: real 0.034s user 0.018s sys 0.014s
I20260619 18:50:49.019579 29786 raft_consensus.cc:348] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:49.020133 29786 raft_consensus.cc:729] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c8ff331acc44d8ad159bd0ae36fa0f, State: Initialized, Role: FOLLOWER
I20260619 18:50:49.020748 29786 consensus_queue.cc:260] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:49.021817 29786 ts_tablet_manager.cc:1428] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
W20260619 18:50:49.046772 29896 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:50:48.994628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:47645)
0619 18:50:48.997198 (+ 2570us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:48.997274 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:48.998421 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:49.045862 (+ 47441us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2295,"thread_start_us":146,"threads_started":1}
W20260619 18:50:49.047271 29894 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:35779 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.254:35779: Client connection negotiation failed: client connection to 127.12.234.254:35779: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 18:50:49.048031 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:49.047769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:35611)
0619 18:50:49.047900 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:49.047968 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 18:50:49.048677 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:49.048471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:47827)
0619 18:50:49.048574 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:49.048628 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 18:50:49.048864 29894 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 18:50:49.111728 29897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:49.112021 29897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:49.112069 29897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:49.116088 29897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:49.116161 29897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:49.116222 29897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:50:49.121491 29897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:36703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=36025
--tserver_master_addrs=127.12.234.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:49.122577 29897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:49.123585 29897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:49.133261 29904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:49.133265 29907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:49.133276 29905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:49.133682 29897 server_base.cc:1034] running on GCE node
I20260619 18:50:49.134152 29897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:49.134840 29897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:49.136020 29897 hybrid_clock.cc:648] HybridClock initialized: now 1781895049136000 us; error 34 us; skew 500 ppm
Jun 19 18:50:49 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895049, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:49.138721 29897 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:50:49.139591 29897 webserver.cc:460] Webserver started at http://127.12.234.194:36025/ using document root <none> and password file <none>
I20260619 18:50:49.140137 29897 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:49.140367 29897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:49.144630 29897 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:50:49.146991 29914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:49.148311 29897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 18:50:49.148427 29897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "e711fc080b1c4dfbb6ca86a84722e25b"
format_stamp: "Formatted at 2026-06-19 18:50:06 on dist-test-slave-fljv"
server_key: "aee7442d1b3cba728d4fe76ff0c0b20f"
server_key_iv: "5742c407a70ba73d0a2fe35f3cb6cbf5"
server_key_version: "encryptionkey@0"
I20260619 18:50:49.149003 29897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:49.175879 29897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:49.179322 29897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:49.179504 29897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:49.180121 29897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:50:49.180958 29897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:50:49.181005 29897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:49.181046 29897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:50:49.181082 29897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:49.189517 29897 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:36703
I20260619 18:50:49.189560 30027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:36703 every 8 connection(s)
I20260619 18:50:49.190593 29897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:50:49.192598 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 29897
I20260619 18:50:49.193075 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:33163
--local_ip_for_outbound_sockets=127.12.234.195
--tserver_master_addrs=127.12.234.254:35779
--webserver_port=37461
--webserver_interface=127.12.234.195
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:50:49.194564 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:49.192865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:44711)
0619 18:50:49.193595 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:49.193730 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":120,"threads_started":1}
W20260619 18:50:49.195022 30028 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:35779 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.254:35779: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
I20260619 18:50:49.195705 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:49.195481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:46069)
0619 18:50:49.195603 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:49.195650 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 18:50:49.196290 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:49.196124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:44671)
0619 18:50:49.196207 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:49.196245 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 18:50:49.196448 30028 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 18:50:49.306821 30031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:50:49.307128 30031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:50:49.307176 30031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:50:49.311690 30031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:50:49.311784 30031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:50:49.311852 30031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:50:49.317317 30031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:33163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=37461
--tserver_master_addrs=127.12.234.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:50:49.318480 30031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:50:49.319541 30031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:50:49.329413 30039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:49.329430 30037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:50:49.329514 30036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:50:49.329840 30031 server_base.cc:1034] running on GCE node
I20260619 18:50:49.330284 30031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:50:49.330974 30031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:50:49.332173 30031 hybrid_clock.cc:648] HybridClock initialized: now 1781895049332145 us; error 38 us; skew 500 ppm
Jun 19 18:50:49 dist-test-slave-fljv krb5kdc[28578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895049, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:50:49.335263 30031 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:50:49.336216 30031 webserver.cc:460] Webserver started at http://127.12.234.195:37461/ using document root <none> and password file <none>
I20260619 18:50:49.336772 30031 fs_manager.cc:359] Metadata directory not provided
I20260619 18:50:49.336834 30031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:50:49.340905 30031 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:50:49.343319 30046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:49.344640 30031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:50:49.344750 30031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "308b15003edc42a48b5f9aecb3fadedb"
format_stamp: "Formatted at 2026-06-19 18:50:07 on dist-test-slave-fljv"
server_key: "30e31d8db5b011ffc166eaf9bf2dfe45"
server_key_iv: "2f229c4e03d1e8bf1f0431b2846cc403"
server_key_version: "encryptionkey@0"
I20260619 18:50:49.345323 30031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:50:49.357491 30031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:50:49.360625 30031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:50:49.360800 30031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:50:49.361414 30031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:50:49.362262 30031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:50:49.362308 30031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:49.362346 30031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:50:49.362362 30031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:50:49.370965 30031 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:33163
I20260619 18:50:49.371039 30159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:33163 every 8 connection(s)
I20260619 18:50:49.372010 30031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:50:49.375871 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:49.374311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:48471)
0619 18:50:49.375042 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:49.375159 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":114,"threads_started":1}
W20260619 18:50:49.376323 30160 heartbeater.cc:644] Failed to heartbeat to 127.12.234.254:35779 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.254:35779: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
I20260619 18:50:49.376926 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:49.376711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:55979)
0619 18:50:49.376818 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:49.376870 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 18:50:49.377488 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:49.377323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:51981)
0619 18:50:49.377404 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:49.377443 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 18:50:49.377696 30160 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 18:50:49.380151 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 30031
I20260619 18:50:50.050022 30164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:50.049545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:41035)
0619 18:50:50.049828 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:50.049898 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":80,"threads_started":1}
I20260619 18:50:50.197424 30165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:50.197045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:39601)
0619 18:50:50.197272 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:50.197324 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
I20260619 18:50:50.206369 30166 raft_consensus.cc:482] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:50:50.206565 30166 raft_consensus.cc:504] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:50.207731 30166 leader_election.cc:288] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703), 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163)
Jun 19 18:50:50 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895048, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.195@KRBTEST.COM
Jun 19 18:50:50 dist-test-slave-fljv krb5kdc[28578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895048, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.194@KRBTEST.COM
I20260619 18:50:50.222005 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:50.208209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:33163 (local address 127.12.234.193:33751)
0619 18:50:50.208508 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:50.208521 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:50.208649 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:50.210122 (+ 1473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:50.210128 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:50.210687 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:50:50.211197 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:50.211212 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:50.212457 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:50.212460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:50.213483 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:50.213493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:50.214201 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:50.214205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:50.214421 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:50.215017 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:50:50.215033 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:50:50.216632 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:50:50.219048 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:50.219054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:50.219070 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:50.219521 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:50.219872 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:50.219875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:50.219877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:50.220064 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:50.220835 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:50:50.220842 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:50:50.221194 (+ 352us) client_negotiation.cc:769] Sending connection context
0619 18:50:50.221425 (+ 231us) client_negotiation.cc:240] Negotiation successful
0619 18:50:50.221838 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":215,"spinlock_wait_cycles":55040,"thread_start_us":82,"threads_started":1}
I20260619 18:50:50.222007 30164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:50.207954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:36703 (local address 127.12.234.193:39629)
0619 18:50:50.208077 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:50.208092 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:50:50.208191 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:50:50.210121 (+ 1930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:50:50.210128 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:50:50.210686 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:50:50.211196 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:50.211212 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:50.212455 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:50.212458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:50.213479 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:50:50.213490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:50:50.214200 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:50:50.214212 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:50:50.214417 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:50.215066 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:50:50.215078 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:50:50.216886 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:50:50.219048 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:50.219054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:50.219070 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:50.219521 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:50.219872 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:50:50.219875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:50:50.219877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:50:50.220064 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:50:50.220835 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:50:50.220842 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:50:50.221194 (+ 352us) client_negotiation.cc:769] Sending connection context
0619 18:50:50.221420 (+ 226us) client_negotiation.cc:240] Negotiation successful
0619 18:50:50.221768 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":231,"spinlock_wait_cycles":33152}
I20260619 18:50:50.222059 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:50.208169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:39629 (local address 127.12.234.194:36703)
0619 18:50:50.208624 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:50.208648 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:50.208699 (+ 51us) server_negotiation.cc:407] Connection header received
0619 18:50:50.209387 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:50.209393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:50.209741 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:50.210025 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:50.211370 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:50.212319 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:50.213602 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:50.214064 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:50.214236 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:50.217135 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:50.217142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:50.217157 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:50.217189 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:50.218889 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:50.219625 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:50.219631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:50.219637 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:50.219783 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:50.220157 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:50.220160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:50.220162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:50.220694 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:50.220823 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:50.221582 (+ 759us) server_negotiation.cc:299] Negotiation successful
0619 18:50:50.221876 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":117,"threads_started":1}
I20260619 18:50:50.222597 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:50.208399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:33751 (local address 127.12.234.195:33163)
0619 18:50:50.208710 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0619 18:50:50.208725 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:50:50.208762 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:50:50.209381 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:50:50.209387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:50:50.209741 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:50:50.210025 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:50:50.211371 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:50.212330 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:50.213600 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:50:50.214064 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:50:50.214240 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:50:50.216862 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:50:50.216867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:50:50.216881 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:50:50.216909 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:50:50.218882 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:50.219627 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:50.219631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:50.219637 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:50.219783 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:50:50.220157 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:50:50.220160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:50:50.220162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:50:50.220695 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:50:50.220823 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 18:50:50.221582 (+ 759us) server_negotiation.cc:299] Negotiation successful
0619 18:50:50.221876 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":86,"threads_started":1}
W20260619 18:50:50.224645 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
W20260619 18:50:50.224866 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
I20260619 18:50:50.224928 29782 leader_election.cc:302] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3c8ff331acc44d8ad159bd0ae36fa0f; no voters: 308b15003edc42a48b5f9aecb3fadedb, e711fc080b1c4dfbb6ca86a84722e25b
I20260619 18:50:50.225199 30166 raft_consensus.cc:2726] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:50:50.378688 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:50.378276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:43539)
0619 18:50:50.378534 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:50.378594 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":74,"threads_started":1}
I20260619 18:50:51.051213 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:51.050789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:33773)
0619 18:50:51.051049 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:51.051115 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260619 18:50:51.198670 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:51.198236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:47177)
0619 18:50:51.198495 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:51.198570 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":69,"threads_started":1}
I20260619 18:50:51.379805 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:51.379370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:34517)
0619 18:50:51.379632 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:51.379697 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260619 18:50:51.831660 30174 raft_consensus.cc:482] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:50:51.831811 30174 raft_consensus.cc:504] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:51.832212 30174 leader_election.cc:288] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703), 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163)
W20260619 18:50:51.833050 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
W20260619 18:50:51.833211 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
I20260619 18:50:51.833272 29782 leader_election.cc:302] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3c8ff331acc44d8ad159bd0ae36fa0f; no voters: 308b15003edc42a48b5f9aecb3fadedb, e711fc080b1c4dfbb6ca86a84722e25b
I20260619 18:50:51.833444 30174 raft_consensus.cc:2726] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:50:52.052397 30175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:52.051937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:57219)
0619 18:50:52.052242 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:52.052301 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":115,"threads_started":1}
I20260619 18:50:52.199927 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:52.199493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:37227)
0619 18:50:52.199764 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:52.199830 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
I20260619 18:50:52.381067 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:52.380657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:50087)
0619 18:50:52.380900 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:52.380964 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
I20260619 18:50:53.053648 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:53.053191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:54379)
0619 18:50:53.053479 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:53.053549 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":102,"threads_started":1}
I20260619 18:50:53.201166 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:53.200695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:57405)
0619 18:50:53.201008 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:53.201071 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260619 18:50:53.382340 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:53.381881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:35673)
0619 18:50:53.382154 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:53.382244 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
I20260619 18:50:53.426105 30181 raft_consensus.cc:482] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:50:53.426268 30181 raft_consensus.cc:504] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:53.426692 30181 leader_election.cc:288] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703), 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163)
W20260619 18:50:53.427520 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
W20260619 18:50:53.427661 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
I20260619 18:50:53.427713 29782 leader_election.cc:302] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3c8ff331acc44d8ad159bd0ae36fa0f; no voters: 308b15003edc42a48b5f9aecb3fadedb, e711fc080b1c4dfbb6ca86a84722e25b
I20260619 18:50:53.427879 30181 raft_consensus.cc:2726] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:50:54.055125 30182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:54.054647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:39825)
0619 18:50:54.054961 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:54.055030 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":72,"threads_started":1}
I20260619 18:50:54.202427 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:54.201952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:44085)
0619 18:50:54.202263 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:54.202324 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260619 18:50:54.383621 30184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:54.383193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:49001)
0619 18:50:54.383455 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:54.383526 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260619 18:50:55.056334 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:55.055851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:57111)
0619 18:50:55.056158 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:55.056234 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":75,"threads_started":1}
I20260619 18:50:55.203773 30186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:55.203252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:52357)
0619 18:50:55.203602 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:55.203675 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":91,"threads_started":1}
I20260619 18:50:55.384910 30187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:55.384485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:34681)
0619 18:50:55.384748 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:55.384814 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260619 18:50:56.057564 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:56.057091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:43395)
0619 18:50:56.057376 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:56.057441 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
I20260619 18:50:56.205176 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:56.204651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:49499)
0619 18:50:56.205006 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:56.205078 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":86,"threads_started":1}
I20260619 18:50:56.338836 30190 raft_consensus.cc:482] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:50:56.338963 30190 raft_consensus.cc:504] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:50:56.339428 30190 leader_election.cc:288] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703), 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163)
W20260619 18:50:56.340210 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
W20260619 18:50:56.340330 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
I20260619 18:50:56.340377 29782 leader_election.cc:302] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3c8ff331acc44d8ad159bd0ae36fa0f; no voters: 308b15003edc42a48b5f9aecb3fadedb, e711fc080b1c4dfbb6ca86a84722e25b
I20260619 18:50:56.340524 30190 raft_consensus.cc:2726] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:50:56.386106 30191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:56.385662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:54911)
0619 18:50:56.385941 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:56.386003 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":67,"threads_started":1}
I20260619 18:50:57.058914 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:57.058439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:57657)
0619 18:50:57.058745 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:57.058815 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":80,"threads_started":1}
I20260619 18:50:57.206673 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:57.206108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:33667)
0619 18:50:57.206496 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:57.206571 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":137,"threads_started":1}
I20260619 18:50:57.387516 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:57.386980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:51133)
0619 18:50:57.387324 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:57.387395 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":83,"threads_started":1}
I20260619 18:50:58.060220 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:58.059739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:47659)
0619 18:50:58.060049 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:58.060121 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":77,"threads_started":1}
I20260619 18:50:58.208150 30196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:58.207605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:55911)
0619 18:50:58.207960 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:58.208043 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":82,"threads_started":1}
I20260619 18:50:58.388883 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:58.388403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:35813)
0619 18:50:58.388708 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:58.388785 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":81,"threads_started":1}
I20260619 18:50:59.061584 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:59.061127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:60273)
0619 18:50:59.061413 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:59.061487 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":88,"threads_started":1}
I20260619 18:50:59.209527 30199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:59.209048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:47169)
0619 18:50:59.209346 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:59.209417 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":81,"threads_started":1}
I20260619 18:50:59.390281 30200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:50:59.389723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:34619)
0619 18:50:59.390077 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 18:50:59.390146 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":122,"threads_started":1}
I20260619 18:51:00.062953 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:00.062499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:34377)
0619 18:51:00.062783 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:00.062851 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":97,"threads_started":1}
I20260619 18:51:00.210785 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:00.210345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:43037)
0619 18:51:00.210613 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:00.210675 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":70,"threads_started":1}
I20260619 18:51:00.391680 30203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:00.391181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:46267)
0619 18:51:00.391503 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:00.391576 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":101,"threads_started":1}
I20260619 18:51:00.703998 30204 raft_consensus.cc:482] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:51:00.704136 30204 raft_consensus.cc:504] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e711fc080b1c4dfbb6ca86a84722e25b" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36703 } } peers { permanent_uuid: "308b15003edc42a48b5f9aecb3fadedb" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33163 } } peers { permanent_uuid: "a3c8ff331acc44d8ad159bd0ae36fa0f" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 37799 } }
I20260619 18:51:00.704550 30204 leader_election.cc:288] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703), 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163)
W20260619 18:51:00.705349 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e711fc080b1c4dfbb6ca86a84722e25b (127.12.234.194:36703): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
W20260619 18:51:00.705483 29782 leader_election.cc:341] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b15003edc42a48b5f9aecb3fadedb (127.12.234.195:33163): Not found: Tablet not found: 681e085b4a4c4728aba6560447be36ab
I20260619 18:51:00.705523 29782 leader_election.cc:302] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3c8ff331acc44d8ad159bd0ae36fa0f; no voters: 308b15003edc42a48b5f9aecb3fadedb, e711fc080b1c4dfbb6ca86a84722e25b
I20260619 18:51:00.705678 30204 raft_consensus.cc:2726] T 681e085b4a4c4728aba6560447be36ab P a3c8ff331acc44d8ad159bd0ae36fa0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 18:51:01.064263 30205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:01.063835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:60031)
0619 18:51:01.064095 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:01.064164 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":81,"threads_started":1}
I20260619 18:51:01.212149 30206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:01.211708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:47729)
0619 18:51:01.211983 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:01.212052 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260619 18:51:01.393218 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:01.392734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:46849)
0619 18:51:01.393029 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:01.393103 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":77,"threads_started":1}
I20260619 18:51:02.065649 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:02.065175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:54513)
0619 18:51:02.065480 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:02.065550 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":95,"threads_started":1}
I20260619 18:51:02.213371 30209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:02.212899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:56237)
0619 18:51:02.213205 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:02.213271 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":89,"threads_started":1}
I20260619 18:51:02.394465 30210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:02.394003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:42959)
0619 18:51:02.394300 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:02.394367 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":83,"threads_started":1}
I20260619 18:51:03.066944 30211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:03.066505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:52267)
0619 18:51:03.066773 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:03.066844 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":80,"threads_started":1}
I20260619 18:51:03.214645 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:03.214156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:35989)
0619 18:51:03.214473 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:03.214526 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
I20260619 18:51:03.395772 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:03.395252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.195:37633)
0619 18:51:03.395575 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:03.395655 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":86,"threads_started":1}
I20260619 18:51:04.068486 30214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:04.067945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.193:36653)
0619 18:51:04.068288 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:04.068353 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":148,"threads_started":1}
I20260619 18:51:04.216089 30215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:04.215578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:35779 (local address 127.12.234.194:54225)
0619 18:51:04.215878 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:04.215949 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.254:35779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":72,"threads_started":1}
I20260619 18:51:04.383504 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 29724
I20260619 18:51:04.388592 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 29897
I20260619 18:51:04.393154 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 30031
2026-06-19T18:51:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59178 ms)
[----------] 30 tests from SecurityITest (237956 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:04 dist-test-slave-fljv krb5kdc[30218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:04 dist-test-slave-fljv krb5kdc[30218](info): set up 2 sockets
Jun 19 18:51:04 dist-test-slave-fljv krb5kdc[30218](info): commencing operation
krb5kdc: starting...
W20260619 18:51:05.361966 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:05 dist-test-slave-fljv krb5kdc[30218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:05.483286 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:43979
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:43979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:05.595386 30231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:05.595676 30231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:05.595719 30231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:05.599828 30231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:05.599896 30231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:05.599915 30231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:05.599931 30231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:05.599944 30231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:05.605438 30231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:43979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:43979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:05.606660 30231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:05.607777 30231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:51:05.616324 30231 server_base.cc:1034] running on GCE node
W20260619 18:51:05.616300 30237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:05.616273 30236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:05.616267 30239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:05.617033 30231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:05.618309 30231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:05.619457 30231 hybrid_clock.cc:648] HybridClock initialized: now 1781895065619434 us; error 34 us; skew 500 ppm
Jun 19 18:51:05 dist-test-slave-fljv krb5kdc[30218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895065, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:05.622157 30231 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:05.623046 30231 webserver.cc:460] Webserver started at http://127.12.234.254:33541/ using document root <none> and password file <none>
I20260619 18:51:05.623562 30231 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:05.623620 30231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:05.623793 30231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:05.625763 30231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "061cce64887d41edbdf8a29de39116c9"
format_stamp: "Formatted at 2026-06-19 18:51:05 on dist-test-slave-fljv"
server_key: "06baa1d6c2ff7215e70f12ab695d0562"
server_key_iv: "721f57cef623786fbc0100694efaec02"
server_key_version: "encryptionkey@0"
I20260619 18:51:05.626297 30231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "061cce64887d41edbdf8a29de39116c9"
format_stamp: "Formatted at 2026-06-19 18:51:05 on dist-test-slave-fljv"
server_key: "06baa1d6c2ff7215e70f12ab695d0562"
server_key_iv: "721f57cef623786fbc0100694efaec02"
server_key_version: "encryptionkey@0"
I20260619 18:51:05.629971 30231 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:05.632462 30246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:05.633713 30231 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:51:05.633831 30231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "061cce64887d41edbdf8a29de39116c9"
format_stamp: "Formatted at 2026-06-19 18:51:05 on dist-test-slave-fljv"
server_key: "06baa1d6c2ff7215e70f12ab695d0562"
server_key_iv: "721f57cef623786fbc0100694efaec02"
server_key_version: "encryptionkey@0"
I20260619 18:51:05.633930 30231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:05.654789 30231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:05.658308 30231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:05.658491 30231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:05.665454 30231 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:43979
I20260619 18:51:05.665552 30298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:43979 every 8 connection(s)
I20260619 18:51:05.666538 30231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:05.669425 30299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:05.669900 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 30231
I20260619 18:51:05.670033 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:05.670388 13227 external_mini_cluster.cc:1428] Setting key 2c908bfce8d5583fcd25388143772f48
I20260619 18:51:05.676890 30299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9: Bootstrap starting.
I20260619 18:51:05.679427 30299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9: Neither blocks nor log segments found. Creating new log.
Jun 19 18:51:05 dist-test-slave-fljv krb5kdc[30218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:05.680210 30299 log.cc:824] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:05.682636 30299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9: No bootstrap required, opened a new log
I20260619 18:51:05.685487 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:05.671743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49610 (local address 127.12.234.254:43979)
0619 18:51:05.672163 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:05.672188 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:05.672233 (+ 45us) server_negotiation.cc:407] Connection header received
0619 18:51:05.672984 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:05.673009 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:05.673447 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:05.673840 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:05.674537 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:05.676047 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:05.676925 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:05.677619 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:05.677773 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:05.680281 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:05.680287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:05.680306 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:05.680341 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:05.682304 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:05.682726 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:05.682732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:05.682738 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:05.682875 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:05.683166 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:05.683169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:05.683171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:05.683695 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:05.683881 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:05.684131 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 18:51:05.684449 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":107,"threads_started":1}
I20260619 18:51:05.687227 30299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061cce64887d41edbdf8a29de39116c9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43979 } }
I20260619 18:51:05.687469 30299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:05.687518 30299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 061cce64887d41edbdf8a29de39116c9, State: Initialized, Role: FOLLOWER
I20260619 18:51:05.687971 30299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061cce64887d41edbdf8a29de39116c9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43979 } }
I20260619 18:51:05.688109 30299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:05.688171 30299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:05.688265 30299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:05.689188 30299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061cce64887d41edbdf8a29de39116c9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43979 } }
I20260619 18:51:05.689538 30299 leader_election.cc:302] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 061cce64887d41edbdf8a29de39116c9; no voters:
I20260619 18:51:05.689829 30299 leader_election.cc:288] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:05.689930 30304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:05.690172 30304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [term 1 LEADER]: Becoming Leader. State: Replica: 061cce64887d41edbdf8a29de39116c9, State: Running, Role: LEADER
I20260619 18:51:05.690526 30304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061cce64887d41edbdf8a29de39116c9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43979 } }
I20260619 18:51:05.691125 30299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:05.692356 30305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "061cce64887d41edbdf8a29de39116c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061cce64887d41edbdf8a29de39116c9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43979 } } }
I20260619 18:51:05.692397 30306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 061cce64887d41edbdf8a29de39116c9. Latest consensus state: current_term: 1 leader_uuid: "061cce64887d41edbdf8a29de39116c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061cce64887d41edbdf8a29de39116c9" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43979 } } }
I20260619 18:51:05.692492 30305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:05.692492 30306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:05.692868 30313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:05.696108 30313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:05.701669 30313 catalog_manager.cc:1347] Generated new cluster ID: a2bf8fc6ff934d96a93ddb6358383aaa
I20260619 18:51:05.701740 30313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:05.711150 30313 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:05.712188 30313 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:05.720973 30313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9: Generated new TSK 0
I20260619 18:51:05.721801 30313 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:51:05.758136 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:43979
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:05.871246 30325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:05.871516 30325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:05.871559 30325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:05.875663 30325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:05.875723 30325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:05.875792 30325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:51:05.881173 30325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:05.882283 30325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:05.883368 30325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:05.893023 30330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:05.893075 30325 server_base.cc:1034] running on GCE node
W20260619 18:51:05.893038 30333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:05.893043 30331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:05.893689 30325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:05.894418 30325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:05.895572 30325 hybrid_clock.cc:648] HybridClock initialized: now 1781895065895541 us; error 45 us; skew 500 ppm
Jun 19 18:51:05 dist-test-slave-fljv krb5kdc[30218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895065, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:05.898379 30325 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:51:05.899245 30325 webserver.cc:460] Webserver started at http://127.12.234.193:41659/ using document root <none> and password file <none>
I20260619 18:51:05.899784 30325 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:05.899842 30325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:05.900012 30325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:05.902043 30325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "06f81c12c6b446b8832e1e9a348e62eb"
format_stamp: "Formatted at 2026-06-19 18:51:05 on dist-test-slave-fljv"
server_key: "0614d930bff4420afd733c0ba39eca3a"
server_key_iv: "467a5ca7b7b603f0d9807590ab49d166"
server_key_version: "encryptionkey@0"
I20260619 18:51:05.902599 30325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "06f81c12c6b446b8832e1e9a348e62eb"
format_stamp: "Formatted at 2026-06-19 18:51:05 on dist-test-slave-fljv"
server_key: "0614d930bff4420afd733c0ba39eca3a"
server_key_iv: "467a5ca7b7b603f0d9807590ab49d166"
server_key_version: "encryptionkey@0"
I20260619 18:51:05.906265 30325 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 18:51:05.908806 30340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:05.910115 30325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:51:05.910260 30325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "06f81c12c6b446b8832e1e9a348e62eb"
format_stamp: "Formatted at 2026-06-19 18:51:05 on dist-test-slave-fljv"
server_key: "0614d930bff4420afd733c0ba39eca3a"
server_key_iv: "467a5ca7b7b603f0d9807590ab49d166"
server_key_version: "encryptionkey@0"
I20260619 18:51:05.910378 30325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:05.947108 30325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:05.950464 30325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:05.950655 30325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:05.951256 30325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:05.952178 30325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:05.952234 30325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:05.952288 30325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:05.952320 30325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:05.962431 30325 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:43293
I20260619 18:51:05.962522 30453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:43293 every 8 connection(s)
I20260619 18:51:05.963372 30325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:51:05.964710 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 30325
I20260619 18:51:05.964828 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:51:05.965121 13227 external_mini_cluster.cc:1428] Setting key 2c3ef31a95de6820d759162189b4e010
Jun 19 18:51:05 dist-test-slave-fljv krb5kdc[30218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895065, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:51:05.977911 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:05.965286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:58651 (local address 127.12.234.254:43979)
0619 18:51:05.965459 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:05.965464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:05.966475 (+ 1011us) server_negotiation.cc:407] Connection header received
0619 18:51:05.967889 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:05.967892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:05.967951 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:05.968038 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:05.969525 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:05.970337 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:05.971492 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:05.971846 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:05.971963 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:05.974625 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:05.974630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:05.974632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:05.974662 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:05.975854 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:05.976321 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:05.976325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:05.976327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:05.976412 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:05.976671 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:05.976672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:05.976674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:05.976868 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:05.976943 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:05.977535 (+ 592us) server_negotiation.cc:299] Negotiation successful
0619 18:51:05.977670 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 18:51:05.978663 30456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:05.965812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43979 (local address 127.12.234.193:58651)
0619 18:51:05.966329 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:05.966384 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:05.967664 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:05.968240 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:05.968250 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:05.968745 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:05.969376 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:05.969391 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:05.970458 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:05.970460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:05.971373 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:05.971382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:05.971996 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:05.971999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:05.972181 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:05.972775 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:05.972790 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:05.974436 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:05.975948 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:05.975952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:05.975965 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:05.976242 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:05.976477 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:05.976479 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:05.976480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:05.976611 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:05.976971 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:05.976978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:05.977296 (+ 318us) client_negotiation.cc:769] Sending connection context
0619 18:51:05.977464 (+ 168us) client_negotiation.cc:240] Negotiation successful
0619 18:51:05.977753 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":132,"threads_started":1}
I20260619 18:51:05.980012 30454 heartbeater.cc:347] Connected to a master server at 127.12.234.254:43979
I20260619 18:51:05.980268 30454 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:05.980858 30454 heartbeater.cc:511] Master 127.12.234.254:43979 requested a full tablet report, sending...
I20260619 18:51:05.982523 30262 ts_manager.cc:194] Registered new tserver with Master: 06f81c12c6b446b8832e1e9a348e62eb (127.12.234.193:43293)
I20260619 18:51:05.983666 30262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:58651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:51:05.993587 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:43979
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:06.104296 30459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:06.104598 30459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:06.104642 30459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:06.108842 30459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:06.108919 30459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:06.108981 30459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:51:06.114631 30459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:06.115792 30459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:06.116860 30459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:06.127067 30465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:06.127111 30467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:06.127084 30464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:06.127120 30459 server_base.cc:1034] running on GCE node
I20260619 18:51:06.127730 30459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:06.128371 30459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:06.129530 30459 hybrid_clock.cc:648] HybridClock initialized: now 1781895066129497 us; error 52 us; skew 500 ppm
Jun 19 18:51:06 dist-test-slave-fljv krb5kdc[30218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895066, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:06.132299 30459 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:51:06.133195 30459 webserver.cc:460] Webserver started at http://127.12.234.194:35193/ using document root <none> and password file <none>
I20260619 18:51:06.133723 30459 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:06.133781 30459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:06.133948 30459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:06.136104 30459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "0b848f8dbc814e7992e388f2a0af4d4e"
format_stamp: "Formatted at 2026-06-19 18:51:06 on dist-test-slave-fljv"
server_key: "24b14c790dec2323f152b50d2cb92cb6"
server_key_iv: "c471bb18813c81b95a8770953caff79e"
server_key_version: "encryptionkey@0"
I20260619 18:51:06.136660 30459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "0b848f8dbc814e7992e388f2a0af4d4e"
format_stamp: "Formatted at 2026-06-19 18:51:06 on dist-test-slave-fljv"
server_key: "24b14c790dec2323f152b50d2cb92cb6"
server_key_iv: "c471bb18813c81b95a8770953caff79e"
server_key_version: "encryptionkey@0"
I20260619 18:51:06.140669 30459 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260619 18:51:06.143399 30474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:06.144820 30459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:06.144946 30459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "0b848f8dbc814e7992e388f2a0af4d4e"
format_stamp: "Formatted at 2026-06-19 18:51:06 on dist-test-slave-fljv"
server_key: "24b14c790dec2323f152b50d2cb92cb6"
server_key_iv: "c471bb18813c81b95a8770953caff79e"
server_key_version: "encryptionkey@0"
I20260619 18:51:06.145095 30459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:06.164556 30459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:06.167830 30459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:06.167995 30459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:06.168601 30459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:06.169486 30459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:06.169534 30459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:06.169574 30459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:06.169593 30459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:06.178275 30459 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:45107
I20260619 18:51:06.178310 30587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:45107 every 8 connection(s)
I20260619 18:51:06.179217 30459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:51:06.179704 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 30459
I20260619 18:51:06.179831 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:51:06.180127 13227 external_mini_cluster.cc:1428] Setting key 0e9b665327c60909db789f270693069c
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
Jun 19 18:51:06 dist-test-slave-fljv krb5kdc[30218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895066, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:51:06.194789 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.181260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:55851 (local address 127.12.234.254:43979)
0619 18:51:06.181393 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:06.181398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:06.182475 (+ 1077us) server_negotiation.cc:407] Connection header received
0619 18:51:06.183982 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:06.183985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:06.184030 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:06.184131 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:06.185705 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.186333 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.188081 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.188315 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.188399 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.191143 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:06.191147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:06.191150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:06.191181 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:06.192666 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:06.193238 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:06.193241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:06.193243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:06.193326 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:06.193660 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:06.193663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:06.193665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:06.193857 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:06.193941 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:06.194500 (+ 559us) server_negotiation.cc:299] Negotiation successful
0619 18:51:06.194625 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:51:06.195811 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.181765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43979 (local address 127.12.234.194:55851)
0619 18:51:06.182342 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:06.182386 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:06.183725 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:06.184380 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:06.184390 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:06.184872 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:06.185549 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.185567 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.186456 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.186459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.187932 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.187945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.188483 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.188487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.188710 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.189304 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:06.189326 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:06.190977 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:06.192772 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:06.192778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:06.192795 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:06.193146 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:06.193410 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:06.193413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:06.193415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:06.193575 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:06.193952 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:06.193959 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:06.194270 (+ 311us) client_negotiation.cc:769] Sending connection context
0619 18:51:06.194448 (+ 178us) client_negotiation.cc:240] Negotiation successful
0619 18:51:06.194802 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":145,"threads_started":1}
I20260619 18:51:06.197046 30588 heartbeater.cc:347] Connected to a master server at 127.12.234.254:43979
I20260619 18:51:06.197320 30588 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:06.197952 30588 heartbeater.cc:511] Master 127.12.234.254:43979 requested a full tablet report, sending...
I20260619 18:51:06.199167 30262 ts_manager.cc:194] Registered new tserver with Master: 0b848f8dbc814e7992e388f2a0af4d4e (127.12.234.194:45107)
I20260619 18:51:06.199780 30262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:55851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:51:06.208948 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:43979
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:06.328631 30593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:06.328953 30593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:06.329058 30593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:06.334966 30593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:06.335057 30593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:06.335150 30593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:51:06.343523 30593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:06.345093 30593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:06.346552 30593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:06.357717 30599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:06.357705 30598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:06.357708 30601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:06.357944 30593 server_base.cc:1034] running on GCE node
I20260619 18:51:06.358508 30593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:06.359230 30593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:06.360399 30593 hybrid_clock.cc:648] HybridClock initialized: now 1781895066360372 us; error 39 us; skew 500 ppm
Jun 19 18:51:06 dist-test-slave-fljv krb5kdc[30218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895066, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:06.363266 30593 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:51:06.364194 30593 webserver.cc:460] Webserver started at http://127.12.234.195:33121/ using document root <none> and password file <none>
I20260619 18:51:06.364730 30593 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:06.364785 30593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:06.364959 30593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:06.367156 30593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "71ab6b16437647af9a9b2b2b69a5c6fe"
format_stamp: "Formatted at 2026-06-19 18:51:06 on dist-test-slave-fljv"
server_key: "240d052227f391a7ba80ddde12e5a34e"
server_key_iv: "44efe16baf2ce2e6cb0d213e333962de"
server_key_version: "encryptionkey@0"
I20260619 18:51:06.367714 30593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "71ab6b16437647af9a9b2b2b69a5c6fe"
format_stamp: "Formatted at 2026-06-19 18:51:06 on dist-test-slave-fljv"
server_key: "240d052227f391a7ba80ddde12e5a34e"
server_key_iv: "44efe16baf2ce2e6cb0d213e333962de"
server_key_version: "encryptionkey@0"
I20260619 18:51:06.371695 30593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:06.374294 30608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:06.375768 30593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:06.375885 30593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "71ab6b16437647af9a9b2b2b69a5c6fe"
format_stamp: "Formatted at 2026-06-19 18:51:06 on dist-test-slave-fljv"
server_key: "240d052227f391a7ba80ddde12e5a34e"
server_key_iv: "44efe16baf2ce2e6cb0d213e333962de"
server_key_version: "encryptionkey@0"
I20260619 18:51:06.375988 30593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:06.410316 30593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:06.413831 30593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:06.414016 30593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:06.414675 30593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:06.415601 30593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:06.415652 30593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:06.415709 30593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:06.415737 30593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:06.424888 30593 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:41585
I20260619 18:51:06.424916 30721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:41585 every 8 connection(s)
I20260619 18:51:06.425828 30593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:51:06.426071 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 30593
I20260619 18:51:06.426237 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:51:06.426594 13227 external_mini_cluster.cc:1428] Setting key 0e272f080dd9bb8d90aaf7f438cf8964
Jun 19 18:51:06 dist-test-slave-fljv krb5kdc[30218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895066, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:06.439404 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.427809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:58027 (local address 127.12.234.254:43979)
0619 18:51:06.428693 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:06.428697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:06.428981 (+ 284us) server_negotiation.cc:407] Connection header received
0619 18:51:06.430367 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:06.430370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:06.430428 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:06.430497 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:06.431712 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.432235 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.433361 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.433581 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.433665 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.436049 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:06.436053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:06.436056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:06.436086 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:06.437337 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:06.437827 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:06.437830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:06.437831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:06.437908 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:06.438282 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:06.438285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:06.438287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:06.438471 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:06.438554 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:06.439101 (+ 547us) server_negotiation.cc:299] Negotiation successful
0619 18:51:06.439229 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778}
I20260619 18:51:06.440394 30724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.428334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43979 (local address 127.12.234.195:58027)
0619 18:51:06.428830 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:06.428888 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:06.430128 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:06.430681 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:06.430689 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:06.431065 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:06.431570 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.431584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.432353 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.432356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.433246 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.433255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.433720 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.433724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.433886 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.434464 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:06.434480 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:06.435881 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:06.437448 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:06.437455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:06.437472 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:06.437733 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:06.437989 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:06.437992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:06.437994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:06.438164 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:06.438586 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:06.438593 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:06.438855 (+ 262us) client_negotiation.cc:769] Sending connection context
0619 18:51:06.439020 (+ 165us) client_negotiation.cc:240] Negotiation successful
0619 18:51:06.439420 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":126,"threads_started":1}
I20260619 18:51:06.441411 30722 heartbeater.cc:347] Connected to a master server at 127.12.234.254:43979
I20260619 18:51:06.441640 30722 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:06.442134 30722 heartbeater.cc:511] Master 127.12.234.254:43979 requested a full tablet report, sending...
I20260619 18:51:06.443109 30262 ts_manager.cc:194] Registered new tserver with Master: 71ab6b16437647af9a9b2b2b69a5c6fe (127.12.234.195:41585)
I20260619 18:51:06.443668 30262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:58027
I20260619 18:51:06.452840 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:51:06.462074 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.454918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.12.234.254:43979)
0619 18:51:06.455038 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:06.455074 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:06.455264 (+ 190us) server_negotiation.cc:407] Connection header received
0619 18:51:06.455393 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:06.455396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:06.455438 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:06.455501 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:06.456216 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.456734 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.457565 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.457786 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.457871 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.459317 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:06.459320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:06.459323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:06.459350 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:06.460490 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:06.460874 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:06.460878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:06.460880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:06.460957 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:06.461193 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:06.461197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:06.461199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:06.461426 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:06.461544 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:06.461791 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 18:51:06.461907 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 18:51:06.465499 30262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:51:06.468734 30262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:51:06.483927 30737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.478405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39582 (local address 127.12.234.194:45107)
0619 18:51:06.478939 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:06.478951 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:06.478975 (+ 24us) server_negotiation.cc:407] Connection header received
0619 18:51:06.479090 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:06.479096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:06.479309 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:06.479466 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:06.479711 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.480742 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.482625 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.483234 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.483355 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.483508 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:06.483665 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 18:51:06.483767 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":145,"threads_started":1}
I20260619 18:51:06.483920 30735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.477690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:45107 (local address 127.0.0.1:39582)
0619 18:51:06.478464 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:06.478476 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:06.478561 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:06.479442 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:06.479445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:06.479459 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:06.479601 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.479608 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.480863 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.480866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.482477 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.482489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.483354 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.483357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.483473 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.483511 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 18:51:06.483625 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 18:51:06.483765 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":56,"thread_start_us":92,"threads_started":1}
I20260619 18:51:06.484246 30736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.478320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:43293 (local address 127.0.0.1:57560)
0619 18:51:06.479215 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:06.479228 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:06.479303 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:06.479910 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:06.479913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:06.479931 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:06.480027 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.480034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.481430 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.481433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.482428 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.482437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.483759 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.483763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.483906 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.483950 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 18:51:06.484018 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 18:51:06.484107 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":25,"thread_start_us":98,"threads_started":1}
I20260619 18:51:06.484333 30734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.477693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.12.234.195:41585)
0619 18:51:06.478556 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:06.478570 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:06.478596 (+ 26us) server_negotiation.cc:407] Connection header received
0619 18:51:06.478676 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:06.478682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:06.478912 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:06.479411 (+ 499us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:06.479757 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.480906 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.482584 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.483436 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.483562 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.483701 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:06.484100 (+ 399us) server_negotiation.cc:299] Negotiation successful
0619 18:51:06.484201 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":91,"threads_started":1}
I20260619 18:51:06.484639 30738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.478751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57560 (local address 127.12.234.193:43293)
0619 18:51:06.479443 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:06.479458 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:06.479478 (+ 20us) server_negotiation.cc:407] Connection header received
0619 18:51:06.479562 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:06.479567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:06.479749 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:06.479888 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:06.480290 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.481300 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.482550 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.483436 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.483562 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.483692 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:06.484379 (+ 687us) server_negotiation.cc:299] Negotiation successful
0619 18:51:06.484492 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":99,"threads_started":1}
I20260619 18:51:06.484953 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.477569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:41585 (local address 127.0.0.1:51838)
0619 18:51:06.477935 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:06.477988 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:06.478115 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:06.479240 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:06.479248 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:06.479560 (+ 312us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:06.479645 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.479651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.481021 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.481024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.482477 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.482489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.483765 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.483773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.483910 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.483959 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 18:51:06.484018 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 18:51:06.484846 (+ 828us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":865,"thread_start_us":92,"threads_started":1}
I20260619 18:51:06.487316 30388 tablet_service.cc:1467] Processing CreateTablet for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9 (DEFAULT_TABLE table=test-table [id=b89b1f645af64f3caa4e2bd0f76525f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:06.487697 30522 tablet_service.cc:1467] Processing CreateTablet for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9 (DEFAULT_TABLE table=test-table [id=b89b1f645af64f3caa4e2bd0f76525f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:06.487805 30656 tablet_service.cc:1467] Processing CreateTablet for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9 (DEFAULT_TABLE table=test-table [id=b89b1f645af64f3caa4e2bd0f76525f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:06.488476 30388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:06.488790 30656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:06.488790 30522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:06.495553 30739 tablet_bootstrap.cc:492] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb: Bootstrap starting.
I20260619 18:51:06.497181 30741 tablet_bootstrap.cc:492] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: Bootstrap starting.
I20260619 18:51:06.497181 30740 tablet_bootstrap.cc:492] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: Bootstrap starting.
I20260619 18:51:06.498234 30739 tablet_bootstrap.cc:654] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:06.499048 30739 log.cc:824] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:06.499612 30740 tablet_bootstrap.cc:654] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:06.499605 30741 tablet_bootstrap.cc:654] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:06.500936 30740 log.cc:824] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:06.500936 30741 log.cc:824] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:06.500994 30739 tablet_bootstrap.cc:492] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb: No bootstrap required, opened a new log
I20260619 18:51:06.501178 30739 ts_tablet_manager.cc:1397] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 18:51:06.503026 30740 tablet_bootstrap.cc:492] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: No bootstrap required, opened a new log
I20260619 18:51:06.503242 30740 ts_tablet_manager.cc:1397] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:51:06.503538 30741 tablet_bootstrap.cc:492] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: No bootstrap required, opened a new log
I20260619 18:51:06.503765 30741 ts_tablet_manager.cc:1397] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 18:51:06.505182 30739 raft_consensus.cc:348] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } }
I20260619 18:51:06.505488 30739 raft_consensus.cc:374] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:06.505568 30739 raft_consensus.cc:729] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06f81c12c6b446b8832e1e9a348e62eb, State: Initialized, Role: FOLLOWER
I20260619 18:51:06.506034 30739 consensus_queue.cc:260] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } }
I20260619 18:51:06.506737 30454 heartbeater.cc:503] Master 127.12.234.254:43979 was elected leader, sending a full tablet report...
I20260619 18:51:06.507149 30739 ts_tablet_manager.cc:1428] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:51:06.507205 30740 raft_consensus.cc:348] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } }
I20260619 18:51:06.507510 30740 raft_consensus.cc:374] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:06.507593 30740 raft_consensus.cc:729] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b848f8dbc814e7992e388f2a0af4d4e, State: Initialized, Role: FOLLOWER
I20260619 18:51:06.507550 30741 raft_consensus.cc:348] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } }
I20260619 18:51:06.507891 30741 raft_consensus.cc:374] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:06.507952 30741 raft_consensus.cc:729] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ab6b16437647af9a9b2b2b69a5c6fe, State: Initialized, Role: FOLLOWER
I20260619 18:51:06.508112 30740 consensus_queue.cc:260] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } }
I20260619 18:51:06.508353 30741 consensus_queue.cc:260] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } }
I20260619 18:51:06.508893 30588 heartbeater.cc:503] Master 127.12.234.254:43979 was elected leader, sending a full tablet report...
I20260619 18:51:06.509204 30741 ts_tablet_manager.cc:1428] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 18:51:06.509204 30740 ts_tablet_manager.cc:1428] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:51:06.509521 30722 heartbeater.cc:503] Master 127.12.234.254:43979 was elected leader, sending a full tablet report...
W20260619 18:51:06.677085 30723 tablet.cc:2369] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:51:06.680410 30589 tablet.cc:2369] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:51:06.714782 30455 tablet.cc:2369] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:06.762204 30746 raft_consensus.cc:482] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:51:06.762432 30746 raft_consensus.cc:504] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } }
I20260619 18:51:06.763605 30746 leader_election.cc:288] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71ab6b16437647af9a9b2b2b69a5c6fe (127.12.234.195:41585), 06f81c12c6b446b8832e1e9a348e62eb (127.12.234.193:43293)
I20260619 18:51:06.767534 30738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.763994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:40433 (local address 127.12.234.193:43293)
0619 18:51:06.764142 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:06.764146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:06.764290 (+ 144us) server_negotiation.cc:407] Connection header received
0619 18:51:06.764430 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:06.764433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:06.764484 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:06.764548 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:06.764947 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.765460 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.766596 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.767006 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.767117 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.767174 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:06.767333 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 18:51:06.767413 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 18:51:06.767549 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.763874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:43293 (local address 127.12.234.194:40433)
0619 18:51:06.764207 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:06.764218 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:06.764329 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:06.764578 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:06.764580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:06.764601 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:06.764724 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.764730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.765563 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.765566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.766473 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.766483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.767117 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.767120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.767216 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.767240 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 18:51:06.767303 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 18:51:06.767384 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":72,"threads_started":1}
I20260619 18:51:06.768007 30734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.763959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:51571 (local address 127.12.234.195:41585)
0619 18:51:06.764160 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:06.764165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:06.764359 (+ 194us) server_negotiation.cc:407] Connection header received
0619 18:51:06.764483 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:06.764485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:06.764536 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:06.764706 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:06.765163 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.765701 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.766809 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.767289 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.767424 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.767484 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:06.767802 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 18:51:06.767886 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260619 18:51:06.768198 30408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d0eb2b8090041e7b15bdaf0fa4ee0a9" candidate_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06f81c12c6b446b8832e1e9a348e62eb" is_pre_election: true
I20260619 18:51:06.768350 30749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.763875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:41585 (local address 127.12.234.194:51571)
0619 18:51:06.764264 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:06.764275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:06.764381 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:06.764719 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:06.764722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:06.764738 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:06.765006 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.765017 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.765822 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.765825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.766683 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:06.766693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.767420 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.767424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:06.767523 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.767544 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 18:51:06.768159 (+ 615us) client_negotiation.cc:240] Negotiation successful
0619 18:51:06.768237 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":36,"thread_start_us":77,"threads_started":1}
I20260619 18:51:06.768549 30408 raft_consensus.cc:2454] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b848f8dbc814e7992e388f2a0af4d4e in term 0.
I20260619 18:51:06.768976 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d0eb2b8090041e7b15bdaf0fa4ee0a9" candidate_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" is_pre_election: true
I20260619 18:51:06.769083 30476 leader_election.cc:302] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06f81c12c6b446b8832e1e9a348e62eb, 0b848f8dbc814e7992e388f2a0af4d4e; no voters:
I20260619 18:51:06.769284 30676 raft_consensus.cc:2454] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b848f8dbc814e7992e388f2a0af4d4e in term 0.
I20260619 18:51:06.769333 30746 raft_consensus.cc:2781] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:51:06.769421 30746 raft_consensus.cc:482] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:51:06.769480 30746 raft_consensus.cc:3037] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:06.770630 30746 raft_consensus.cc:504] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } }
I20260619 18:51:06.770973 30746 leader_election.cc:288] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [CANDIDATE]: Term 1 election: Requested vote from peers 71ab6b16437647af9a9b2b2b69a5c6fe (127.12.234.195:41585), 06f81c12c6b446b8832e1e9a348e62eb (127.12.234.193:43293)
I20260619 18:51:06.771281 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d0eb2b8090041e7b15bdaf0fa4ee0a9" candidate_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe"
I20260619 18:51:06.771317 30408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d0eb2b8090041e7b15bdaf0fa4ee0a9" candidate_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06f81c12c6b446b8832e1e9a348e62eb"
I20260619 18:51:06.771396 30676 raft_consensus.cc:3037] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:06.771443 30408 raft_consensus.cc:3037] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:06.772418 30676 raft_consensus.cc:2454] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b848f8dbc814e7992e388f2a0af4d4e in term 1.
I20260619 18:51:06.772526 30408 raft_consensus.cc:2454] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b848f8dbc814e7992e388f2a0af4d4e in term 1.
I20260619 18:51:06.772771 30475 leader_election.cc:302] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b848f8dbc814e7992e388f2a0af4d4e, 71ab6b16437647af9a9b2b2b69a5c6fe; no voters:
I20260619 18:51:06.773007 30746 raft_consensus.cc:2781] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:06.773221 30746 raft_consensus.cc:686] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 1 LEADER]: Becoming Leader. State: Replica: 0b848f8dbc814e7992e388f2a0af4d4e, State: Running, Role: LEADER
I20260619 18:51:06.773552 30746 consensus_queue.cc:237] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } }
I20260619 18:51:06.776743 30260 catalog_manager.cc:5495] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b848f8dbc814e7992e388f2a0af4d4e (127.12.234.194). New cstate: current_term: 1 leader_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b848f8dbc814e7992e388f2a0af4d4e" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 } health_report { overall_health: UNKNOWN } } }
I20260619 18:51:06.852427 30737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.848870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.12.234.194:45107)
0619 18:51:06.848991 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:06.848995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:06.849010 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:51:06.849105 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:06.849108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:06.849165 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:06.849267 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:06.849521 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.850045 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.850893 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:06.851128 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:06.851210 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:06.851568 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:06.852016 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:06.852162 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:06.852218 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 18:51:06.852273 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:51:06.858774 30676 raft_consensus.cc:1261] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [term 1 FOLLOWER]: Refusing update from remote peer 0b848f8dbc814e7992e388f2a0af4d4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:06.858775 30408 raft_consensus.cc:1261] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb [term 1 FOLLOWER]: Refusing update from remote peer 0b848f8dbc814e7992e388f2a0af4d4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:06.859464 30746 consensus_queue.cc:1035] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "06f81c12c6b446b8832e1e9a348e62eb" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:06.859688 30750 consensus_queue.cc:1035] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "71ab6b16437647af9a9b2b2b69a5c6fe" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 41585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:06.868597 30753 mvcc.cc:205] Tried to move back new op lower bound from 7298642193845813248 to 7298642193507115008. Current Snapshot: MvccSnapshot[applied={T|T < 7298642193845813248 or (T in {7298642193845813248})}]
I20260619 18:51:06.871007 30755 mvcc.cc:205] Tried to move back new op lower bound from 7298642193845813248 to 7298642193507115008. Current Snapshot: MvccSnapshot[applied={T|T < 7298642193845813248}]
I20260619 18:51:06.875522 30260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:06.875715 30260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:06.878228 30260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 061cce64887d41edbdf8a29de39116c9: Sending DeleteTablet for 3 replicas of tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9
I20260619 18:51:06.878973 30388 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:06 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:57560
I20260619 18:51:06.879057 30656 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:06 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:51838
I20260619 18:51:06.879508 30763 tablet_replica.cc:330] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: stopping tablet replica
I20260619 18:51:06.879067 30655 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:06 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:51838
I20260619 18:51:06.879807 30522 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:06 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:39582
I20260619 18:51:06.880149 30762 tablet_replica.cc:330] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 06f81c12c6b446b8832e1e9a348e62eb: stopping tablet replica
I20260619 18:51:06.880375 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 30325
I20260619 18:51:06.881124 30763 raft_consensus.cc:2229] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:51:06.881498 30763 raft_consensus.cc:2258] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:06.882040 30764 tablet_replica.cc:330] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: stopping tablet replica
I20260619 18:51:06.882349 30764 raft_consensus.cc:2229] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:51:06.882719 30764 raft_consensus.cc:2258] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:06.883484 30763 ts_tablet_manager.cc:1905] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:06.884047 30764 ts_tablet_manager.cc:1905] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:06.886408 30763 ts_tablet_manager.cc:1918] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:06.886590 30763 log.cc:1196] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/4d0eb2b8090041e7b15bdaf0fa4ee0a9
I20260619 18:51:06.886691 30764 ts_tablet_manager.cc:1918] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:06.886780 30764 log.cc:1196] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/4d0eb2b8090041e7b15bdaf0fa4ee0a9
I20260619 18:51:06.887070 30764 ts_tablet_manager.cc:1939] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 0b848f8dbc814e7992e388f2a0af4d4e: Deleting consensus metadata
I20260619 18:51:06.887082 30763 ts_tablet_manager.cc:1939] T 4d0eb2b8090041e7b15bdaf0fa4ee0a9 P 71ab6b16437647af9a9b2b2b69a5c6fe: Deleting consensus metadata
W20260619 18:51:06.887576 30248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:43293 (error 108)
I20260619 18:51:06.887780 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 30459
I20260619 18:51:06.889391 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.889048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:43293 (local address 127.0.0.1:57574)
0619 18:51:06.889216 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:06.889299 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:43293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 18:51:06.889950 30247 catalog_manager.cc:4841] TS 71ab6b16437647af9a9b2b2b69a5c6fe (127.12.234.195:41585): tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9 (table test-table [id=b89b1f645af64f3caa4e2bd0f76525f6]) successfully deleted
W20260619 18:51:06.890319 30248 catalog_manager.cc:4584] TS 06f81c12c6b446b8832e1e9a348e62eb (127.12.234.193:43293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9: Network error: Client connection negotiation failed: client connection to 127.12.234.193:43293: connect: Connection refused (error 111)
W20260619 18:51:06.890401 30247 catalog_manager.cc:4808] TS 71ab6b16437647af9a9b2b2b69a5c6fe (127.12.234.195:41585): delete failed for tablet 4d0eb2b8090041e7b15bdaf0fa4ee0a9 because the tablet was not found. No further retry: Not found: Tablet not found: 4d0eb2b8090041e7b15bdaf0fa4ee0a9
I20260619 18:51:06.893396 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 30593
I20260619 18:51:06.895231 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:06.893554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:45107 (local address 127.0.0.1:39608)
0619 18:51:06.895098 (+ 1544us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:06.895154 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.194:45107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1456}
I20260619 18:51:06.898942 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 30231
2026-06-19T18:51:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2505 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T18:51:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:06Z Disabled control of system clock
I20260619 18:51:06.927606 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:39369
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:39369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 18:51:07.039129 30769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:07.039426 30769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:07.039463 30769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:07.043809 30769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:07.043883 30769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:07.043897 30769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:07.043913 30769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:07.043928 30769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:07.049088 30769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:39369
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:39369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:07.050323 30769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:07.051790 30769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:07.060463 30777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:07.060490 30774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:07.060490 30775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:07.060860 30769 server_base.cc:1034] running on GCE node
I20260619 18:51:07.061463 30769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:07.062713 30769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:07.063911 30769 hybrid_clock.cc:648] HybridClock initialized: now 1781895067063897 us; error 38 us; skew 500 ppm
I20260619 18:51:07.066013 30769 webserver.cc:460] Webserver started at http://127.12.234.254:45949/ using document root <none> and password file <none>
I20260619 18:51:07.066632 30769 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:07.066695 30769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:07.066903 30769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:07.069053 30769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "dbec447d0be54cb097c219d7bb9e5149"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "f5c729bbfc41c0d4a0ae8ad97f8980f9"
server_key_iv: "f06ec6f966f165409ec1ca24f4ea4619"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.069598 30769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "dbec447d0be54cb097c219d7bb9e5149"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "f5c729bbfc41c0d4a0ae8ad97f8980f9"
server_key_iv: "f06ec6f966f165409ec1ca24f4ea4619"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.073460 30769 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 18:51:07.076068 30783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.077528 30769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:51:07.077653 30769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "dbec447d0be54cb097c219d7bb9e5149"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "f5c729bbfc41c0d4a0ae8ad97f8980f9"
server_key_iv: "f06ec6f966f165409ec1ca24f4ea4619"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.077782 30769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:07.095734 30769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:07.096484 30769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:07.096660 30769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:07.103567 30769 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:39369
I20260619 18:51:07.103590 30835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:39369 every 8 connection(s)
I20260619 18:51:07.104564 30769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:07.107808 30836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:07.113713 30836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149: Bootstrap starting.
I20260619 18:51:07.113730 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 30769
I20260619 18:51:07.113842 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:07.114210 13227 external_mini_cluster.cc:1428] Setting key dfed0391d66beafe8a84a0f355a3aad3
I20260619 18:51:07.116509 30836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:07.117439 30836 log.cc:824] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:07.119563 30836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149: No bootstrap required, opened a new log
I20260619 18:51:07.123370 30836 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbec447d0be54cb097c219d7bb9e5149" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39369 } }
I20260619 18:51:07.123622 30836 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:07.123678 30836 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbec447d0be54cb097c219d7bb9e5149, State: Initialized, Role: FOLLOWER
I20260619 18:51:07.124150 30836 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbec447d0be54cb097c219d7bb9e5149" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39369 } }
I20260619 18:51:07.124250 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.115720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.12.234.254:39369)
0619 18:51:07.116429 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.116449 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.116498 (+ 49us) server_negotiation.cc:407] Connection header received
0619 18:51:07.117422 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.117449 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.118003 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.118376 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:07.118834 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.119869 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.120755 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.121322 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.121482 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.122363 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:07.122368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:07.122381 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:07.122781 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:07.122844 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:07.122865 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:07.123039 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.123114 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.123380 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":113,"threads_started":1}
I20260619 18:51:07.124285 30836 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:07.124349 30836 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:07.124445 30836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:07.125402 30836 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbec447d0be54cb097c219d7bb9e5149" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39369 } }
I20260619 18:51:07.125768 30836 leader_election.cc:302] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbec447d0be54cb097c219d7bb9e5149; no voters:
I20260619 18:51:07.126111 30836 leader_election.cc:288] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:07.126247 30841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:07.126490 30841 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [term 1 LEADER]: Becoming Leader. State: Replica: dbec447d0be54cb097c219d7bb9e5149, State: Running, Role: LEADER
I20260619 18:51:07.126791 30841 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbec447d0be54cb097c219d7bb9e5149" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39369 } }
I20260619 18:51:07.127380 30836 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:07.128297 30842 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbec447d0be54cb097c219d7bb9e5149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbec447d0be54cb097c219d7bb9e5149" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39369 } } }
I20260619 18:51:07.128443 30842 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:07.128676 30841 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbec447d0be54cb097c219d7bb9e5149. Latest consensus state: current_term: 1 leader_uuid: "dbec447d0be54cb097c219d7bb9e5149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbec447d0be54cb097c219d7bb9e5149" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39369 } } }
I20260619 18:51:07.128764 30841 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:07.128875 30850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:07.132140 30850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:07.138157 30850 catalog_manager.cc:1347] Generated new cluster ID: 17d8b5ffbb7e454db7a8277218ca402b
I20260619 18:51:07.138269 30850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:07.155130 30850 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:07.156481 30850 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:07.164849 30850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149: Generated new TSK 0
I20260619 18:51:07.165701 30850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:07.175544 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:39369
--builtin_ntp_servers=127.12.234.212:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 18:51:07.310678 30860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:07.310995 30860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:07.311035 30860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:07.315235 30860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:07.315313 30860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:07.315371 30860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:51:07.320485 30860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:39369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:07.321712 30860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:07.323227 30860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:07.333585 30865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:07.333623 30866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:07.333630 30868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:07.333817 30860 server_base.cc:1034] running on GCE node
I20260619 18:51:07.334437 30860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:07.335129 30860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:07.336300 30860 hybrid_clock.cc:648] HybridClock initialized: now 1781895067336277 us; error 37 us; skew 500 ppm
I20260619 18:51:07.338485 30860 webserver.cc:460] Webserver started at http://127.12.234.193:33257/ using document root <none> and password file <none>
I20260619 18:51:07.339097 30860 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:07.339162 30860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:07.339358 30860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:07.341598 30860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "40be4348779b47f8a8b166922d1ebf8e"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "b2abf54a391ed867b36718b7db594f1b"
server_key_iv: "f04ff96f6fe327c58aa73f1a9b9ac8bc"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.342211 30860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "40be4348779b47f8a8b166922d1ebf8e"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "b2abf54a391ed867b36718b7db594f1b"
server_key_iv: "f04ff96f6fe327c58aa73f1a9b9ac8bc"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.346457 30860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:51:07.349066 30874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.350531 30860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:07.350658 30860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "40be4348779b47f8a8b166922d1ebf8e"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "b2abf54a391ed867b36718b7db594f1b"
server_key_iv: "f04ff96f6fe327c58aa73f1a9b9ac8bc"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.350783 30860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:07.364673 30860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:07.365450 30860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:07.365649 30860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:07.366318 30860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:07.367344 30860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:07.367394 30860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.367452 30860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:07.367478 30860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.376497 30860 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:39331
I20260619 18:51:07.376514 30987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:39331 every 8 connection(s)
I20260619 18:51:07.377489 30860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:51:07.383450 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 30860
I20260619 18:51:07.383562 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:51:07.383845 13227 external_mini_cluster.cc:1428] Setting key 9881df601334f24d994d329df1736531
I20260619 18:51:07.386027 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:39369
--builtin_ntp_servers=127.12.234.212:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 18:51:07.387873 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.379390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:54693 (local address 127.12.234.254:39369)
0619 18:51:07.379518 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.379522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.380720 (+ 1198us) server_negotiation.cc:407] Connection header received
0619 18:51:07.382313 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.382317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.382404 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.382481 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:07.383535 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.384142 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.385653 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.385883 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.385964 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.387329 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:07.387332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:07.387334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:07.387348 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:07.387370 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:07.387378 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:07.387458 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.387649 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.387720 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:51:07.388947 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.379818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:39369 (local address 127.12.234.193:54693)
0619 18:51:07.380521 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:07.380593 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:07.382043 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:07.382719 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:07.382728 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:07.382838 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:07.383378 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.383391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.384284 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.384288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.385529 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.385542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.386047 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.386052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.386319 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.386969 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:51:07.386972 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:07.387039 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:51:07.387136 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:07.387485 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:07.387498 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:07.387503 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 18:51:07.387640 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 18:51:07.388003 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":110,"threads_started":1}
I20260619 18:51:07.390609 30988 heartbeater.cc:347] Connected to a master server at 127.12.234.254:39369
I20260619 18:51:07.390880 30988 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:07.391528 30988 heartbeater.cc:511] Master 127.12.234.254:39369 requested a full tablet report, sending...
I20260619 18:51:07.393517 30800 ts_manager.cc:194] Registered new tserver with Master: 40be4348779b47f8a8b166922d1ebf8e (127.12.234.193:39331)
I20260619 18:51:07.395310 30800 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.193:54693
W20260619 18:51:07.501323 30991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:07.501606 30991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:07.501653 30991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:07.505718 30991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:07.505800 30991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:07.505854 30991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:51:07.510696 30991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:39369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:07.511793 30991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:07.513232 30991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:51:07.522960 30991 server_base.cc:1034] running on GCE node
W20260619 18:51:07.522897 30997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:07.523005 30996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:07.522897 30999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:07.523586 30991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:07.524251 30991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:07.525439 30991 hybrid_clock.cc:648] HybridClock initialized: now 1781895067525421 us; error 41 us; skew 500 ppm
I20260619 18:51:07.527477 30991 webserver.cc:460] Webserver started at http://127.12.234.194:32973/ using document root <none> and password file <none>
I20260619 18:51:07.528082 30991 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:07.528149 30991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:07.528378 30991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:07.530689 30991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "775a1eb79aeb4c72b44e6e11593043b0"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "34c24a7acbc51eda12b551c5dad8aa73"
server_key_iv: "e70b6338c57109de39c375cd5c922928"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.531294 30991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "775a1eb79aeb4c72b44e6e11593043b0"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "34c24a7acbc51eda12b551c5dad8aa73"
server_key_iv: "e70b6338c57109de39c375cd5c922928"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.535423 30991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:07.538096 31005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.539525 30991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:07.539656 30991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "775a1eb79aeb4c72b44e6e11593043b0"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "34c24a7acbc51eda12b551c5dad8aa73"
server_key_iv: "e70b6338c57109de39c375cd5c922928"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.539770 30991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:07.561465 30991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:07.562278 30991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:07.562460 30991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:07.563094 30991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:07.564074 30991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:07.564121 30991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.564174 30991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:07.564203 30991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.572800 30991 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:45791
I20260619 18:51:07.572808 31118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:45791 every 8 connection(s)
I20260619 18:51:07.573761 30991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:51:07.583086 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 30991
I20260619 18:51:07.583175 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.575640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:53925 (local address 127.12.234.254:39369)
0619 18:51:07.575764 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.575768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.577030 (+ 1262us) server_negotiation.cc:407] Connection header received
0619 18:51:07.578511 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.578514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.578587 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.578660 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:07.579611 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.580129 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.581301 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.581538 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.581652 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.582721 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:07.582724 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:07.582727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:07.582742 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:07.582761 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:07.582769 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:07.582826 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.582960 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.583026 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 18:51:07.583206 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:51:07.583534 13227 external_mini_cluster.cc:1428] Setting key 1ee86050e1ef34f0389f7beff0f28059
I20260619 18:51:07.584101 31121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.576211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:39369 (local address 127.12.234.194:53925)
0619 18:51:07.576884 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:07.576925 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:07.578288 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:07.578857 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:07.578865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:07.578959 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:07.579470 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.579483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.580246 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.580249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.581188 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.581198 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.581692 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.581696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.581856 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.582466 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:51:07.582468 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:07.582510 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:51:07.582579 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:07.582835 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:07.582844 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:07.582850 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 18:51:07.582959 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 18:51:07.583259 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":117,"threads_started":1}
I20260619 18:51:07.585247 31119 heartbeater.cc:347] Connected to a master server at 127.12.234.254:39369
I20260619 18:51:07.585469 31119 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:07.585597 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:39369
--builtin_ntp_servers=127.12.234.212:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 18:51:07.585995 31119 heartbeater.cc:511] Master 127.12.234.254:39369 requested a full tablet report, sending...
I20260619 18:51:07.587101 30799 ts_manager.cc:194] Registered new tserver with Master: 775a1eb79aeb4c72b44e6e11593043b0 (127.12.234.194:45791)
I20260619 18:51:07.587679 30799 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.194:53925
W20260619 18:51:07.698256 31122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:07.698590 31122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:07.698648 31122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:07.703297 31122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:07.703390 31122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:07.703483 31122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:51:07.708869 31122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:39369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:07.710067 31122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:07.711558 31122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:07.721642 31127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:07.721841 31130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:07.721695 31122 server_base.cc:1034] running on GCE node
W20260619 18:51:07.721642 31128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:07.722528 31122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:07.723237 31122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:07.724403 31122 hybrid_clock.cc:648] HybridClock initialized: now 1781895067724382 us; error 34 us; skew 500 ppm
I20260619 18:51:07.726333 31122 webserver.cc:460] Webserver started at http://127.12.234.195:35985/ using document root <none> and password file <none>
I20260619 18:51:07.726902 31122 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:07.726967 31122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:07.727170 31122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:07.729414 31122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "bda39a9f0b6045209247de62eb36ec28"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "73cb73ea8a3331981d2c78f30b0d9a4e"
server_key_iv: "a8a4c1a534c422bee01d8c303ebdcd96"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.729980 31122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "bda39a9f0b6045209247de62eb36ec28"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "73cb73ea8a3331981d2c78f30b0d9a4e"
server_key_iv: "a8a4c1a534c422bee01d8c303ebdcd96"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.733788 31122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:51:07.736370 31136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.737650 31122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 18:51:07.737772 31122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "bda39a9f0b6045209247de62eb36ec28"
format_stamp: "Formatted at 2026-06-19 18:51:07 on dist-test-slave-fljv"
server_key: "73cb73ea8a3331981d2c78f30b0d9a4e"
server_key_iv: "a8a4c1a534c422bee01d8c303ebdcd96"
server_key_version: "encryptionkey@0"
I20260619 18:51:07.737884 31122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:07.754447 31122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:07.755257 31122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:07.755457 31122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:07.756119 31122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:07.757138 31122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:07.757193 31122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.757238 31122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:07.757268 31122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:07.765965 31122 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:46779
I20260619 18:51:07.766007 31249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:46779 every 8 connection(s)
I20260619 18:51:07.766947 31122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:51:07.772279 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 31122
I20260619 18:51:07.772403 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:51:07.772729 13227 external_mini_cluster.cc:1428] Setting key 59e159c0a0191bb2370652d92127b064
I20260619 18:51:07.776976 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.768831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:58889 (local address 127.12.234.254:39369)
0619 18:51:07.769003 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.769008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.770091 (+ 1083us) server_negotiation.cc:407] Connection header received
0619 18:51:07.771591 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.771610 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.771669 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.771746 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:07.773101 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.773770 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.775155 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.775375 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.775456 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.776569 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:07.776571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:07.776574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:07.776586 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:07.776605 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:07.776614 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:07.776672 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.776803 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.776858 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 18:51:07.777887 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.769240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:39369 (local address 127.12.234.195:58889)
0619 18:51:07.769927 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:07.769974 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:07.771338 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:07.771997 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:07.772007 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:07.772133 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:07.772907 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.772929 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.773897 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.773901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.775046 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.775055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.775554 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.775557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.775720 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.776288 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:51:07.776290 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:07.776333 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:51:07.776406 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:07.776684 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:07.776690 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:07.776694 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 18:51:07.776790 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 18:51:07.777047 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":102,"threads_started":1}
I20260619 18:51:07.779028 31250 heartbeater.cc:347] Connected to a master server at 127.12.234.254:39369
I20260619 18:51:07.779279 31250 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:07.779819 31250 heartbeater.cc:511] Master 127.12.234.254:39369 requested a full tablet report, sending...
I20260619 18:51:07.780891 30799 ts_manager.cc:194] Registered new tserver with Master: bda39a9f0b6045209247de62eb36ec28 (127.12.234.195:46779)
I20260619 18:51:07.781477 30799 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.195:58889
I20260619 18:51:07.786749 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:51:07.792313 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.788729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.12.234.254:39369)
0619 18:51:07.788868 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.788872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.789038 (+ 166us) server_negotiation.cc:407] Connection header received
0619 18:51:07.789150 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.789152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.789194 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.789265 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:07.789499 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.790032 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.790900 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.791133 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.791216 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.791925 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:07.791929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:07.791932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:07.791946 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:07.791971 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:07.791979 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:07.792039 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.792138 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.792195 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 18:51:07.795305 30799 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:51:07.798501 30799 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:51:07.813148 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.807339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.12.234.195:46779)
0619 18:51:07.808402 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.808413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.808436 (+ 23us) server_negotiation.cc:407] Connection header received
0619 18:51:07.808794 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.808800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.808970 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.809117 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:07.809326 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.810449 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.811672 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.812433 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.812561 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.812768 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.812849 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.812967 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":154,"threads_started":1}
I20260619 18:51:07.813197 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.807139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:46779 (local address 127.0.0.1:35874)
0619 18:51:07.807949 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:07.807963 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:07.808110 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:07.809103 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:07.809106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:07.809122 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:07.809222 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.809229 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.810568 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.810571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.811542 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.811552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.812561 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.812564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.812672 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.812705 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 18:51:07.812816 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 18:51:07.813024 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":3,"thread_start_us":111,"threads_started":1}
I20260619 18:51:07.813248 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.807346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.12.234.194:45791)
0619 18:51:07.807793 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.807806 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.808042 (+ 236us) server_negotiation.cc:407] Connection header received
0619 18:51:07.808247 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.808253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.808476 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.808660 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:07.808953 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.810126 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.811934 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.812706 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.812822 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.812980 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.813024 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.813108 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":127,"threads_started":1}
I20260619 18:51:07.813196 31261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.806998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:45791 (local address 127.0.0.1:52292)
0619 18:51:07.807879 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:07.807936 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:07.808093 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:07.808655 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:07.808660 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:07.808680 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:07.808834 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.808841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.810252 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.810256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.811794 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.811806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.812827 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.812830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.812908 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.812927 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 18:51:07.812982 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 18:51:07.813038 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":93,"thread_start_us":74,"threads_started":1}
I20260619 18:51:07.814980 31262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.807736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:39331 (local address 127.0.0.1:56938)
0619 18:51:07.808543 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:07.808554 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:07.808654 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:07.809257 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:07.809259 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:07.809274 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:07.809344 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.809349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.810347 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.810350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.811796 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.811806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.814592 (+ 2786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.814596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.814683 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.814705 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:51:07.814771 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 18:51:07.814844 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":97,"thread_start_us":82,"threads_started":1}
I20260619 18:51:07.815399 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.808128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56938 (local address 127.12.234.193:39331)
0619 18:51:07.808525 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.808543 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.808633 (+ 90us) server_negotiation.cc:407] Connection header received
0619 18:51:07.808802 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.808808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.809002 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.809129 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:07.809441 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.810174 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.813454 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.814235 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.814347 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.814524 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.815146 (+ 622us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.815252 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":126,"threads_started":1}
I20260619 18:51:07.816759 31053 tablet_service.cc:1467] Processing CreateTablet for tablet 23fa47f744b24455a419f25ebc9ac1eb (DEFAULT_TABLE table=test-table [id=0166955063984c29a7d026febc175a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:07.816769 31184 tablet_service.cc:1467] Processing CreateTablet for tablet 23fa47f744b24455a419f25ebc9ac1eb (DEFAULT_TABLE table=test-table [id=0166955063984c29a7d026febc175a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:07.817814 31184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23fa47f744b24455a419f25ebc9ac1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:07.817816 31053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23fa47f744b24455a419f25ebc9ac1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:07.818373 30922 tablet_service.cc:1467] Processing CreateTablet for tablet 23fa47f744b24455a419f25ebc9ac1eb (DEFAULT_TABLE table=test-table [id=0166955063984c29a7d026febc175a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:07.819401 30922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23fa47f744b24455a419f25ebc9ac1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:07.824375 31267 tablet_bootstrap.cc:492] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0: Bootstrap starting.
I20260619 18:51:07.826078 31268 tablet_bootstrap.cc:492] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: Bootstrap starting.
I20260619 18:51:07.826323 31269 tablet_bootstrap.cc:492] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e: Bootstrap starting.
I20260619 18:51:07.827024 31267 tablet_bootstrap.cc:654] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:07.827958 31267 log.cc:824] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:07.828071 31269 tablet_bootstrap.cc:654] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:07.828315 31268 tablet_bootstrap.cc:654] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:07.828841 31269 log.cc:824] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:07.829195 31268 log.cc:824] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:07.830821 31269 tablet_bootstrap.cc:492] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e: No bootstrap required, opened a new log
I20260619 18:51:07.830925 31268 tablet_bootstrap.cc:492] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: No bootstrap required, opened a new log
I20260619 18:51:07.830926 31267 tablet_bootstrap.cc:492] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0: No bootstrap required, opened a new log
I20260619 18:51:07.831023 31269 ts_tablet_manager.cc:1397] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 18:51:07.831107 31267 ts_tablet_manager.cc:1397] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 18:51:07.831107 31268 ts_tablet_manager.cc:1397] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260619 18:51:07.834153 31269 raft_consensus.cc:348] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } }
I20260619 18:51:07.834424 31269 raft_consensus.cc:374] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:07.834477 31269 raft_consensus.cc:729] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40be4348779b47f8a8b166922d1ebf8e, State: Initialized, Role: FOLLOWER
I20260619 18:51:07.834905 31269 consensus_queue.cc:260] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } }
I20260619 18:51:07.835216 31268 raft_consensus.cc:348] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } }
I20260619 18:51:07.835216 31267 raft_consensus.cc:348] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } }
I20260619 18:51:07.835475 31268 raft_consensus.cc:374] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:07.835475 31267 raft_consensus.cc:374] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:07.835522 31267 raft_consensus.cc:729] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 775a1eb79aeb4c72b44e6e11593043b0, State: Initialized, Role: FOLLOWER
I20260619 18:51:07.835530 31268 raft_consensus.cc:729] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda39a9f0b6045209247de62eb36ec28, State: Initialized, Role: FOLLOWER
I20260619 18:51:07.835932 31268 consensus_queue.cc:260] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } }
I20260619 18:51:07.835914 31269 ts_tablet_manager.cc:1428] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 18:51:07.835932 31267 consensus_queue.cc:260] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } }
I20260619 18:51:07.836431 30988 heartbeater.cc:503] Master 127.12.234.254:39369 was elected leader, sending a full tablet report...
I20260619 18:51:07.836948 31267 ts_tablet_manager.cc:1428] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:51:07.837003 31268 ts_tablet_manager.cc:1428] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:51:07.837311 31250 heartbeater.cc:503] Master 127.12.234.254:39369 was elected leader, sending a full tablet report...
I20260619 18:51:07.838301 31119 heartbeater.cc:503] Master 127.12.234.254:39369 was elected leader, sending a full tablet report...
W20260619 18:51:07.878675 30989 tablet.cc:2369] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:07.921538 31274 raft_consensus.cc:482] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:51:07.921737 31274 raft_consensus.cc:504] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } }
I20260619 18:51:07.923036 31274 leader_election.cc:288] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 775a1eb79aeb4c72b44e6e11593043b0 (127.12.234.194:45791), 40be4348779b47f8a8b166922d1ebf8e (127.12.234.193:39331)
I20260619 18:51:07.927695 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.923256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:45791 (local address 127.12.234.195:51283)
0619 18:51:07.923362 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:07.923378 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:07.923469 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:07.923801 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:07.923804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:07.923828 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:07.923967 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.923975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.924990 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.924993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.926271 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.926282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.927230 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.927235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.927379 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.927407 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 18:51:07.927473 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 18:51:07.927553 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"spinlock_wait_cycles":4224}
I20260619 18:51:07.927839 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.923455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:51283 (local address 127.12.234.194:45791)
0619 18:51:07.923590 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.923594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.923605 (+ 11us) server_negotiation.cc:407] Connection header received
0619 18:51:07.923651 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.923653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.923691 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.923762 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:07.924109 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.924874 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.926419 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.927082 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.927207 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.927263 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.927516 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.927636 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:51:07.928637 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.923392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:50505 (local address 127.12.234.193:39331)
0619 18:51:07.923512 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.923518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.923799 (+ 281us) server_negotiation.cc:407] Connection header received
0619 18:51:07.924008 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.924012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.924070 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.924154 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:07.924541 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.925326 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.926720 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.927356 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.927467 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.927508 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.928431 (+ 923us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.928517 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 18:51:07.928660 31073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23fa47f744b24455a419f25ebc9ac1eb" candidate_uuid: "bda39a9f0b6045209247de62eb36ec28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775a1eb79aeb4c72b44e6e11593043b0" is_pre_election: true
I20260619 18:51:07.929037 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.923324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:39331 (local address 127.12.234.195:50505)
0619 18:51:07.923715 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:07.923726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:07.923861 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:07.924201 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:07.924203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:07.924222 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:07.924359 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.924365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.925465 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.925468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.926602 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:07.926611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.928187 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.928191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:07.928286 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.928311 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 18:51:07.928784 (+ 473us) client_negotiation.cc:240] Negotiation successful
0619 18:51:07.928902 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"spinlock_wait_cycles":384,"thread_start_us":115,"threads_started":1}
I20260619 18:51:07.929015 31073 raft_consensus.cc:2454] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda39a9f0b6045209247de62eb36ec28 in term 0.
I20260619 18:51:07.929616 31140 leader_election.cc:302] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 775a1eb79aeb4c72b44e6e11593043b0, bda39a9f0b6045209247de62eb36ec28; no voters:
I20260619 18:51:07.929921 31274 raft_consensus.cc:2781] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:51:07.930058 31274 raft_consensus.cc:482] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:51:07.930105 31274 raft_consensus.cc:3037] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:07.930981 30942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23fa47f744b24455a419f25ebc9ac1eb" candidate_uuid: "bda39a9f0b6045209247de62eb36ec28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40be4348779b47f8a8b166922d1ebf8e" is_pre_election: true
I20260619 18:51:07.931322 30942 raft_consensus.cc:2454] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda39a9f0b6045209247de62eb36ec28 in term 0.
I20260619 18:51:07.931414 31274 raft_consensus.cc:504] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } }
I20260619 18:51:07.931790 31274 leader_election.cc:288] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [CANDIDATE]: Term 1 election: Requested vote from peers 775a1eb79aeb4c72b44e6e11593043b0 (127.12.234.194:45791), 40be4348779b47f8a8b166922d1ebf8e (127.12.234.193:39331)
I20260619 18:51:07.932245 30942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23fa47f744b24455a419f25ebc9ac1eb" candidate_uuid: "bda39a9f0b6045209247de62eb36ec28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40be4348779b47f8a8b166922d1ebf8e"
I20260619 18:51:07.932312 31073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23fa47f744b24455a419f25ebc9ac1eb" candidate_uuid: "bda39a9f0b6045209247de62eb36ec28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775a1eb79aeb4c72b44e6e11593043b0"
I20260619 18:51:07.932363 30942 raft_consensus.cc:3037] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:07.932443 31073 raft_consensus.cc:3037] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:07.933573 30942 raft_consensus.cc:2454] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda39a9f0b6045209247de62eb36ec28 in term 1.
I20260619 18:51:07.933607 31073 raft_consensus.cc:2454] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda39a9f0b6045209247de62eb36ec28 in term 1.
I20260619 18:51:07.933869 31140 leader_election.cc:302] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 775a1eb79aeb4c72b44e6e11593043b0, bda39a9f0b6045209247de62eb36ec28; no voters:
I20260619 18:51:07.934042 31274 raft_consensus.cc:2781] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:07.934276 31274 raft_consensus.cc:686] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 1 LEADER]: Becoming Leader. State: Replica: bda39a9f0b6045209247de62eb36ec28, State: Running, Role: LEADER
I20260619 18:51:07.934593 31274 consensus_queue.cc:237] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } }
I20260619 18:51:07.937851 30800 catalog_manager.cc:5495] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 reported cstate change: term changed from 0 to 1, leader changed from <none> to bda39a9f0b6045209247de62eb36ec28 (127.12.234.195). New cstate: current_term: 1 leader_uuid: "bda39a9f0b6045209247de62eb36ec28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bda39a9f0b6045209247de62eb36ec28" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 46779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 } health_report { overall_health: UNKNOWN } } }
I20260619 18:51:07.967454 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:07.963889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.12.234.195:46779)
0619 18:51:07.964012 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:07.964016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:07.964031 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:51:07.964164 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:07.964166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:07.964222 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:07.964314 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:07.964580 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.965097 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.965959 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:07.966223 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:07.966314 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:07.966631 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:07.967081 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:07.967191 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:07.967248 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 18:51:07.967310 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 18:51:07.973639 30942 raft_consensus.cc:1261] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e [term 1 FOLLOWER]: Refusing update from remote peer bda39a9f0b6045209247de62eb36ec28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:07.973639 31073 raft_consensus.cc:1261] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [term 1 FOLLOWER]: Refusing update from remote peer bda39a9f0b6045209247de62eb36ec28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:07.974341 31277 consensus_queue.cc:1035] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40be4348779b47f8a8b166922d1ebf8e" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:07.974527 31274 consensus_queue.cc:1035] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "775a1eb79aeb4c72b44e6e11593043b0" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 45791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:07.984355 31282 mvcc.cc:205] Tried to move back new op lower bound from 7298642198412410880 to 7298642198262677504. Current Snapshot: MvccSnapshot[applied={T|T < 7298642198412410880}]
I20260619 18:51:07.990399 30798 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:07.990598 30798 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:07.993127 30798 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbec447d0be54cb097c219d7bb9e5149: Sending DeleteTablet for 3 replicas of tablet 23fa47f744b24455a419f25ebc9ac1eb
I20260619 18:51:07.993909 31053 tablet_service.cc:1514] Processing DeleteTablet for tablet 23fa47f744b24455a419f25ebc9ac1eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:07 UTC) from {username='slave'} at 127.0.0.1:52292
I20260619 18:51:07.993971 30922 tablet_service.cc:1514] Processing DeleteTablet for tablet 23fa47f744b24455a419f25ebc9ac1eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:07 UTC) from {username='slave'} at 127.0.0.1:56938
I20260619 18:51:07.993968 31184 tablet_service.cc:1514] Processing DeleteTablet for tablet 23fa47f744b24455a419f25ebc9ac1eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:07 UTC) from {username='slave'} at 127.0.0.1:35874
I20260619 18:51:07.994445 31290 tablet_replica.cc:330] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0: stopping tablet replica
I20260619 18:51:07.994680 31292 tablet_replica.cc:330] T 23fa47f744b24455a419f25ebc9ac1eb P 40be4348779b47f8a8b166922d1ebf8e: stopping tablet replica
I20260619 18:51:07.994972 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 30860
I20260619 18:51:07.996058 31291 tablet_replica.cc:330] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: stopping tablet replica
I20260619 18:51:07.996366 31291 raft_consensus.cc:2229] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:51:07.996744 31291 raft_consensus.cc:2258] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:08.000110 31291 ts_tablet_manager.cc:1905] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:08.002480 31291 ts_tablet_manager.cc:1918] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:08.002584 31291 log.cc:1196] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/23fa47f744b24455a419f25ebc9ac1eb
I20260619 18:51:08.002590 31290 raft_consensus.cc:2229] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:51:08.002862 31291 ts_tablet_manager.cc:1939] T 23fa47f744b24455a419f25ebc9ac1eb P bda39a9f0b6045209247de62eb36ec28: Deleting consensus metadata
I20260619 18:51:08.003010 31290 raft_consensus.cc:2258] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:08.004274 30785 catalog_manager.cc:4841] TS bda39a9f0b6045209247de62eb36ec28 (127.12.234.195:46779): tablet 23fa47f744b24455a419f25ebc9ac1eb (table test-table [id=0166955063984c29a7d026febc175a3a]) successfully deleted
I20260619 18:51:08.004644 31290 ts_tablet_manager.cc:1905] T 23fa47f744b24455a419f25ebc9ac1eb P 775a1eb79aeb4c72b44e6e11593043b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:08.006101 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 30991
W20260619 18:51:08.006167 30785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:39331 (error 108)
I20260619 18:51:08.011234 31262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:08.010876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:39331 (local address 127.0.0.1:56940)
0619 18:51:08.011028 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:08.011128 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:39331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 18:51:08.011456 30785 catalog_manager.cc:4584] TS 40be4348779b47f8a8b166922d1ebf8e (127.12.234.193:39331): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23fa47f744b24455a419f25ebc9ac1eb: Network error: Client connection negotiation failed: client connection to 127.12.234.193:39331: connect: Connection refused (error 111)
I20260619 18:51:08.012692 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 31122
I20260619 18:51:08.012882 31262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:08.012647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:45791 (local address 127.0.0.1:52300)
0619 18:51:08.012758 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:08.012808 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.194:45791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 18:51:08.018077 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 30769
2026-06-19T18:51:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1119 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:08 dist-test-slave-fljv krb5kdc[31296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:08 dist-test-slave-fljv krb5kdc[31296](info): set up 2 sockets
Jun 19 18:51:08 dist-test-slave-fljv krb5kdc[31296](info): commencing operation
krb5kdc: starting...
W20260619 18:51:08.974133 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:09 dist-test-slave-fljv krb5kdc[31296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:09.097747 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:36233
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:36233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:09.211206 31309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:09.211504 31309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:09.211545 31309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:09.215584 31309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:09.215656 31309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:09.215675 31309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:09.215691 31309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:09.215705 31309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:09.221314 31309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:36233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:36233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:09.222450 31309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:09.223484 31309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:09.231992 31317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:09.232046 31314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:09.232030 31315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:09.232372 31309 server_base.cc:1034] running on GCE node
I20260619 18:51:09.232985 31309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:09.234251 31309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:09.235430 31309 hybrid_clock.cc:648] HybridClock initialized: now 1781895069235405 us; error 38 us; skew 500 ppm
Jun 19 18:51:09 dist-test-slave-fljv krb5kdc[31296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:09.238440 31309 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:09.239403 31309 webserver.cc:460] Webserver started at http://127.12.234.254:40167/ using document root <none> and password file <none>
I20260619 18:51:09.239964 31309 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:09.240028 31309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:09.240222 31309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:09.242408 31309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "bc560a59204a47feb65c897fc3457dd1"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "6963f6e8a9af71f3c63f7af2f660bf7d"
server_key_iv: "4ac30c32a33194220dc03da3a96684cc"
server_key_version: "encryptionkey@0"
I20260619 18:51:09.242959 31309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "bc560a59204a47feb65c897fc3457dd1"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "6963f6e8a9af71f3c63f7af2f660bf7d"
server_key_iv: "4ac30c32a33194220dc03da3a96684cc"
server_key_version: "encryptionkey@0"
I20260619 18:51:09.246899 31309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 18:51:09.249571 31324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:09.250980 31309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:09.251164 31309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "bc560a59204a47feb65c897fc3457dd1"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "6963f6e8a9af71f3c63f7af2f660bf7d"
server_key_iv: "4ac30c32a33194220dc03da3a96684cc"
server_key_version: "encryptionkey@0"
I20260619 18:51:09.251291 31309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:09.262252 31309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:09.265470 31309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:09.265663 31309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:09.272987 31309 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:36233
I20260619 18:51:09.273025 31376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:36233 every 8 connection(s)
I20260619 18:51:09.274067 31309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:09.274453 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 31309
I20260619 18:51:09.274586 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:09.274902 13227 external_mini_cluster.cc:1428] Setting key 4349dcc283855bd9ec1550d8dc4a9557
I20260619 18:51:09.277511 31377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 18:51:09 dist-test-slave-fljv krb5kdc[31296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:09.284513 31377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1: Bootstrap starting.
I20260619 18:51:09.286966 31377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:09.287889 31377 log.cc:824] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:09.289973 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:09.276386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.12.234.254:36233)
0619 18:51:09.276820 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:09.276835 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:09.276873 (+ 38us) server_negotiation.cc:407] Connection header received
0619 18:51:09.277557 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:09.277581 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:09.277952 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:09.278313 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:09.279227 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.280272 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.281471 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.281961 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.282081 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:09.284972 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:09.284978 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:09.284992 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:09.285026 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:09.286750 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:09.287205 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:09.287213 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:09.287220 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:09.287378 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:09.287621 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:09.287624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:09.287626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:09.288197 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:09.288392 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:09.288660 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 18:51:09.288971 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":138,"threads_started":1}
I20260619 18:51:09.290308 31377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1: No bootstrap required, opened a new log
I20260619 18:51:09.294124 31377 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc560a59204a47feb65c897fc3457dd1" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36233 } }
I20260619 18:51:09.294432 31377 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:09.294473 31377 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc560a59204a47feb65c897fc3457dd1, State: Initialized, Role: FOLLOWER
I20260619 18:51:09.294883 31377 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc560a59204a47feb65c897fc3457dd1" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36233 } }
I20260619 18:51:09.294996 31377 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:09.295042 31377 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:09.295116 31377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:09.296149 31377 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc560a59204a47feb65c897fc3457dd1" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36233 } }
I20260619 18:51:09.296481 31377 leader_election.cc:302] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc560a59204a47feb65c897fc3457dd1; no voters:
I20260619 18:51:09.296787 31377 leader_election.cc:288] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:09.296918 31382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:09.297149 31382 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [term 1 LEADER]: Becoming Leader. State: Replica: bc560a59204a47feb65c897fc3457dd1, State: Running, Role: LEADER
I20260619 18:51:09.297461 31382 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc560a59204a47feb65c897fc3457dd1" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36233 } }
I20260619 18:51:09.298081 31377 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:09.299240 31384 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc560a59204a47feb65c897fc3457dd1. Latest consensus state: current_term: 1 leader_uuid: "bc560a59204a47feb65c897fc3457dd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc560a59204a47feb65c897fc3457dd1" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36233 } } }
I20260619 18:51:09.299377 31384 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:09.299584 31383 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc560a59204a47feb65c897fc3457dd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc560a59204a47feb65c897fc3457dd1" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 36233 } } }
I20260619 18:51:09.299736 31383 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:09.299793 31391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:09.303200 31391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:09.309504 31391 catalog_manager.cc:1347] Generated new cluster ID: ffe76c912c2d40bd8aa72d2353b716a6
I20260619 18:51:09.309592 31391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:09.323269 31391 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:09.325042 31391 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:09.330302 31391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1: Generated new TSK 0
I20260619 18:51:09.331141 31391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:51:09.370553 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:36233
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:09.521077 31403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:09.521384 31403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:09.521432 31403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:09.525553 31403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:09.525641 31403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:09.525718 31403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:51:09.531420 31403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:36233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:09.532655 31403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:09.533788 31403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:09.544371 31408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:09.544409 31411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:09.544497 31409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:09.544566 31403 server_base.cc:1034] running on GCE node
I20260619 18:51:09.545171 31403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:09.545897 31403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:09.547067 31403 hybrid_clock.cc:648] HybridClock initialized: now 1781895069547026 us; error 51 us; skew 500 ppm
Jun 19 18:51:09 dist-test-slave-fljv krb5kdc[31296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:09.555436 31403 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:51:09.556495 31403 webserver.cc:460] Webserver started at http://127.12.234.193:43177/ using document root <none> and password file <none>
I20260619 18:51:09.557068 31403 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:09.557132 31403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:09.557312 31403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:09.559541 31403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "c929954e6a46df77dedf59a550878d5b"
server_key_iv: "53a4b3f625da3064b4db22698f0114a9"
server_key_version: "encryptionkey@0"
I20260619 18:51:09.560117 31403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "c929954e6a46df77dedf59a550878d5b"
server_key_iv: "53a4b3f625da3064b4db22698f0114a9"
server_key_version: "encryptionkey@0"
I20260619 18:51:09.564144 31403 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:51:09.566898 31418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:09.568274 31403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:09.568392 31403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "c929954e6a46df77dedf59a550878d5b"
server_key_iv: "53a4b3f625da3064b4db22698f0114a9"
server_key_version: "encryptionkey@0"
I20260619 18:51:09.568523 31403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:09.586110 31403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:09.589830 31403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:09.590020 31403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:09.590701 31403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:09.591588 31403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:09.591636 31403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:09.591676 31403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:09.591691 31403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:09.600327 31403 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:35463
I20260619 18:51:09.600363 31531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:35463 every 8 connection(s)
I20260619 18:51:09.601295 31403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:51:09.608668 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 31403
I20260619 18:51:09.608819 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:51:09.609124 13227 external_mini_cluster.cc:1428] Setting key e303bf64406cf55df4f5738f7aada771
Jun 19 18:51:09 dist-test-slave-fljv krb5kdc[31296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:09.615980 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:09.603215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:52123 (local address 127.12.234.254:36233)
0619 18:51:09.603386 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:09.603390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:09.604323 (+ 933us) server_negotiation.cc:407] Connection header received
0619 18:51:09.605656 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:09.605659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:09.605712 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:09.605793 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:09.607212 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.607768 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.609300 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.609564 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.609671 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:09.612498 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:09.612502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:09.612505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:09.612536 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:09.613882 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:09.614405 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:09.614408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:09.614409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:09.614489 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:09.614828 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:09.614830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:09.614832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:09.615008 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:09.615155 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:09.615665 (+ 510us) server_negotiation.cc:299] Negotiation successful
0619 18:51:09.615809 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 18:51:09.616816 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:09.603621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:36233 (local address 127.12.234.193:52123)
0619 18:51:09.604154 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:09.604202 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:09.605412 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:09.606020 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:09.606028 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:09.606456 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:09.607048 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:09.607065 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.607881 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.607883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:09.609181 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:09.609194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.609732 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.609737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:09.609931 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:09.610584 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:09.610605 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:09.612297 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:09.613993 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:09.613999 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:09.614013 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:09.614314 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:09.614594 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:09.614596 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:09.614597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:09.614732 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:09.615159 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:09.615167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:09.615423 (+ 256us) client_negotiation.cc:769] Sending connection context
0619 18:51:09.615595 (+ 172us) client_negotiation.cc:240] Negotiation successful
0619 18:51:09.615950 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:51:09.618088 31532 heartbeater.cc:347] Connected to a master server at 127.12.234.254:36233
I20260619 18:51:09.618351 31532 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:09.618876 31532 heartbeater.cc:511] Master 127.12.234.254:36233 requested a full tablet report, sending...
I20260619 18:51:09.620558 31339 ts_manager.cc:194] Registered new tserver with Master: 6caadb549a7d4e109fe12f3e9c5bd3ef (127.12.234.193:35463)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:51:09.621805 31339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:52123
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:51:09.637353 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:36233
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:09.750627 31537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:09.750919 31537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:09.750964 31537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:09.755215 31537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:09.755301 31537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:09.755378 31537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:51:09.761023 31537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:36233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:09.762151 31537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:09.763248 31537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:09.773581 31543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:09.773576 31545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:09.773579 31542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:09.773964 31537 server_base.cc:1034] running on GCE node
I20260619 18:51:09.774528 31537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:09.775252 31537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:09.776420 31537 hybrid_clock.cc:648] HybridClock initialized: now 1781895069776397 us; error 36 us; skew 500 ppm
Jun 19 18:51:09 dist-test-slave-fljv krb5kdc[31296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:09.779431 31537 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:51:09.780427 31537 webserver.cc:460] Webserver started at http://127.12.234.194:35823/ using document root <none> and password file <none>
I20260619 18:51:09.781001 31537 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:09.781070 31537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:09.781266 31537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:09.783419 31537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "662ffc8af1bf40f9af94356c13c44cec"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "bb7dd9baf8c1a8b913f6a3ee9d3090a7"
server_key_iv: "338a5c41ed53a04cd4d0c1b5e74a7cc1"
server_key_version: "encryptionkey@0"
I20260619 18:51:09.783967 31537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "662ffc8af1bf40f9af94356c13c44cec"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "bb7dd9baf8c1a8b913f6a3ee9d3090a7"
server_key_iv: "338a5c41ed53a04cd4d0c1b5e74a7cc1"
server_key_version: "encryptionkey@0"
I20260619 18:51:09.787776 31537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:51:09.790352 31552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:09.791708 31537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:51:09.791838 31537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "662ffc8af1bf40f9af94356c13c44cec"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "bb7dd9baf8c1a8b913f6a3ee9d3090a7"
server_key_iv: "338a5c41ed53a04cd4d0c1b5e74a7cc1"
server_key_version: "encryptionkey@0"
I20260619 18:51:09.791981 31537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:09.807168 31537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:09.810580 31537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:09.810761 31537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:09.811349 31537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:09.812218 31537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:09.812268 31537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:09.812320 31537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:09.812346 31537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:09.821136 31537 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:37217
I20260619 18:51:09.821166 31665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:37217 every 8 connection(s)
I20260619 18:51:09.822111 31537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:51:09.823882 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 31537
I20260619 18:51:09.824095 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:51:09.824409 13227 external_mini_cluster.cc:1428] Setting key 9157f390d2eb829339dc89c4b71aba8d
Jun 19 18:51:09 dist-test-slave-fljv krb5kdc[31296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:51:09.838438 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:09.824134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:54001 (local address 127.12.234.254:36233)
0619 18:51:09.824285 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:09.824289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:09.825410 (+ 1121us) server_negotiation.cc:407] Connection header received
0619 18:51:09.827004 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:09.827008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:09.827065 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:09.827147 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:09.828818 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.829470 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.831310 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.831534 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.831629 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:09.834657 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:09.834660 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:09.834662 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:09.834692 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:09.836073 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:09.836681 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:09.836685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:09.836687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:09.836788 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:09.837141 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:09.837144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:09.837146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:09.837395 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:09.837500 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:09.838087 (+ 587us) server_negotiation.cc:299] Negotiation successful
0619 18:51:09.838271 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 18:51:09.839588 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:09.824671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:36233 (local address 127.12.234.194:54001)
0619 18:51:09.825237 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:09.825301 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:09.826778 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:09.827320 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:09.827328 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:09.827853 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:09.828613 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:09.828636 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.829628 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.829632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:09.831177 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:09.831190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:09.831717 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:09.831722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:09.831966 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:09.832642 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:09.832664 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:09.834452 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:09.836255 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:09.836261 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:09.836276 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:09.836569 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:09.836865 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:09.836868 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:09.836870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:09.837046 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:09.837519 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:09.837527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:09.837823 (+ 296us) client_negotiation.cc:769] Sending connection context
0619 18:51:09.837996 (+ 173us) client_negotiation.cc:240] Negotiation successful
0619 18:51:09.838454 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":133,"threads_started":1}
I20260619 18:51:09.840850 31666 heartbeater.cc:347] Connected to a master server at 127.12.234.254:36233
I20260619 18:51:09.841152 31666 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:09.841818 31666 heartbeater.cc:511] Master 127.12.234.254:36233 requested a full tablet report, sending...
I20260619 18:51:09.843214 31339 ts_manager.cc:194] Registered new tserver with Master: 662ffc8af1bf40f9af94356c13c44cec (127.12.234.194:37217)
I20260619 18:51:09.844040 31339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:54001
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:51:09.854079 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:36233
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:44267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:09.971755 31671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:09.972084 31671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:09.972133 31671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:09.976647 31671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:09.976727 31671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:09.976809 31671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:51:09.982760 31671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:44267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:36233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:09.983964 31671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:09.985045 31671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:09.995353 31677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:09.995362 31679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:09.995362 31676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:09.995611 31671 server_base.cc:1034] running on GCE node
I20260619 18:51:09.996193 31671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:09.996937 31671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:09.998137 31671 hybrid_clock.cc:648] HybridClock initialized: now 1781895069998111 us; error 36 us; skew 500 ppm
Jun 19 18:51:09 dist-test-slave-fljv krb5kdc[31296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:10.001135 31671 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:51:10.002048 31671 webserver.cc:460] Webserver started at http://127.12.234.195:36045/ using document root <none> and password file <none>
I20260619 18:51:10.002660 31671 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:10.002725 31671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:10.002897 31671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:10.004997 31671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "95e298dd7588f5805d6b878e10f5a523"
server_key_iv: "bdfd9939c4155dae817a596067884d60"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.005527 31671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "95e298dd7588f5805d6b878e10f5a523"
server_key_iv: "bdfd9939c4155dae817a596067884d60"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.009490 31671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:51:10.012055 31686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:10.013362 31671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:51:10.013469 31671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff"
format_stamp: "Formatted at 2026-06-19 18:51:09 on dist-test-slave-fljv"
server_key: "95e298dd7588f5805d6b878e10f5a523"
server_key_iv: "bdfd9939c4155dae817a596067884d60"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.013589 31671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:10.033223 31671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:10.036887 31671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:10.037071 31671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:10.037662 31671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:10.038615 31671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:10.038664 31671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:10.038704 31671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:10.038718 31671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:10.047277 31671 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:33667
I20260619 18:51:10.047302 31799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:33667 every 8 connection(s)
I20260619 18:51:10.048290 31671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:51:10.051141 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 31671
I20260619 18:51:10.051295 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:51:10.051662 13227 external_mini_cluster.cc:1428] Setting key bfc8b2f75fa2dfaa7741ada43adf8f09
Jun 19 18:51:10 dist-test-slave-fljv krb5kdc[31296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:10.063947 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.050227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:37299 (local address 127.12.234.254:36233)
0619 18:51:10.050372 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.050375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.051399 (+ 1024us) server_negotiation.cc:407] Connection header received
0619 18:51:10.053062 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.053066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.053123 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.053197 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:10.055119 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.055653 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.056897 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.057135 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.057224 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.060062 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:10.060066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:10.060068 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:10.060097 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:10.061685 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:10.062326 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:10.062329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:10.062331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:10.062417 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:10.062781 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:10.062783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:10.062785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:10.062979 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:10.063070 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.063633 (+ 563us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.063769 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 18:51:10.064908 31802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.050655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:36233 (local address 127.12.234.195:37299)
0619 18:51:10.051211 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:10.051281 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:10.052829 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:10.053899 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:10.053908 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:10.054408 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:10.054967 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.054984 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.055780 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.055783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.056768 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.056779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.057286 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.057290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.057472 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.058116 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:10.058135 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:10.059866 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:10.061831 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:10.061839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:10.061859 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:10.062215 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:10.062507 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:10.062510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:10.062512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:10.062683 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:10.063091 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:10.063098 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:10.063376 (+ 278us) client_negotiation.cc:769] Sending connection context
0619 18:51:10.063559 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 18:51:10.063932 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":120,"threads_started":1}
I20260619 18:51:10.066145 31800 heartbeater.cc:347] Connected to a master server at 127.12.234.254:36233
I20260619 18:51:10.066445 31800 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:10.067001 31800 heartbeater.cc:511] Master 127.12.234.254:36233 requested a full tablet report, sending...
I20260619 18:51:10.068115 31339 ts_manager.cc:194] Registered new tserver with Master: 7b8ae07b0f7b4b1581c1a5c25cea90ff (127.12.234.195:33667)
I20260619 18:51:10.068719 31339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:37299
I20260619 18:51:10.078531 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:51:10.088704 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.080626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58752 (local address 127.12.234.254:36233)
0619 18:51:10.080842 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.080846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.080933 (+ 87us) server_negotiation.cc:407] Connection header received
0619 18:51:10.081091 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.081094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.081145 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.081211 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:10.081993 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.082607 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.083471 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.083712 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.083801 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.085404 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:10.085409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:10.085411 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:10.085443 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:10.086814 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:10.087367 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:10.087370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:10.087371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:10.087452 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:10.087770 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:10.087774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:10.087776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:10.088017 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:10.088154 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.088439 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.088531 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260619 18:51:10.092073 31339 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:51:10.095427 31339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:51:10.111011 31816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.105504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.12.234.195:33667)
0619 18:51:10.106458 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.106469 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.106489 (+ 20us) server_negotiation.cc:407] Connection header received
0619 18:51:10.106557 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.106562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.106735 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.106886 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:10.107086 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.108016 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.109358 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.110287 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.110407 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.110540 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.110758 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.110853 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":129,"threads_started":1}
I20260619 18:51:10.110975 31811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.104465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:33667 (local address 127.0.0.1:43358)
0619 18:51:10.104811 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:10.104862 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:10.104995 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:10.106874 (+ 1879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:10.106877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:10.106896 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:10.106973 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.106979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.108139 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.108142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.109254 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.109263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.110404 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.110407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.110519 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.110570 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 18:51:10.110687 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 18:51:10.110810 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":32,"spinlock_wait_cycles":5248,"thread_start_us":89,"threads_started":1}
I20260619 18:51:10.111711 31812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.104520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:35463 (local address 127.0.0.1:46136)
0619 18:51:10.104811 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:10.104862 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:10.104995 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:10.106062 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:10.106068 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:10.106092 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:10.106290 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.106297 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.107701 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.107704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.109337 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.109348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.111289 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.111293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.111390 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.111414 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 18:51:10.111507 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 18:51:10.111573 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":145,"spinlock_wait_cycles":6400,"thread_start_us":121,"threads_started":1}
I20260619 18:51:10.111783 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.105300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46136 (local address 127.12.234.193:35463)
0619 18:51:10.105561 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.105577 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.105600 (+ 23us) server_negotiation.cc:407] Connection header received
0619 18:51:10.105679 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.105684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.105909 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.106079 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:10.106406 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.107569 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.110273 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.111131 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.111290 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.111417 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.111554 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.111654 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":67,"threads_started":1}
I20260619 18:51:10.112618 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.105213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:37217 (local address 127.0.0.1:42650)
0619 18:51:10.105862 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:10.105874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:10.105965 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:10.106532 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:10.106535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:10.106550 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:10.106630 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.106638 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.107728 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.107730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.109338 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.109348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.112215 (+ 2867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.112218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.112301 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.112325 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 18:51:10.112405 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 18:51:10.112478 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":158,"thread_start_us":91,"threads_started":1}
I20260619 18:51:10.112699 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.105393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.12.234.194:37217)
0619 18:51:10.105804 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.105815 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.105954 (+ 139us) server_negotiation.cc:407] Connection header received
0619 18:51:10.106200 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.106206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.106400 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.106536 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:10.106733 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.107574 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.109441 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.110287 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.110407 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.110540 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.112457 (+ 1917us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.112567 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":89,"threads_started":1}
I20260619 18:51:10.114478 31734 tablet_service.cc:1467] Processing CreateTablet for tablet c4295675b15a4ebea5bea8a84b629bb3 (DEFAULT_TABLE table=test-table [id=edca2f9defae4a5ea8c2c319f079e037]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:10.114941 31466 tablet_service.cc:1467] Processing CreateTablet for tablet c4295675b15a4ebea5bea8a84b629bb3 (DEFAULT_TABLE table=test-table [id=edca2f9defae4a5ea8c2c319f079e037]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:10.115506 31734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4295675b15a4ebea5bea8a84b629bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:10.115477 31600 tablet_service.cc:1467] Processing CreateTablet for tablet c4295675b15a4ebea5bea8a84b629bb3 (DEFAULT_TABLE table=test-table [id=edca2f9defae4a5ea8c2c319f079e037]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:10.116010 31466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4295675b15a4ebea5bea8a84b629bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:10.116454 31600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4295675b15a4ebea5bea8a84b629bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:10.121884 31817 tablet_bootstrap.cc:492] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: Bootstrap starting.
I20260619 18:51:10.124146 31819 tablet_bootstrap.cc:492] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: Bootstrap starting.
I20260619 18:51:10.124701 31817 tablet_bootstrap.cc:654] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:10.125749 31817 log.cc:824] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:10.125984 31819 tablet_bootstrap.cc:654] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:10.126971 31819 log.cc:824] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:10.127513 31818 tablet_bootstrap.cc:492] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef: Bootstrap starting.
I20260619 18:51:10.127756 31817 tablet_bootstrap.cc:492] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: No bootstrap required, opened a new log
I20260619 18:51:10.127955 31817 ts_tablet_manager.cc:1397] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 18:51:10.128747 31819 tablet_bootstrap.cc:492] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: No bootstrap required, opened a new log
I20260619 18:51:10.129007 31819 ts_tablet_manager.cc:1397] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260619 18:51:10.129815 31818 tablet_bootstrap.cc:654] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:10.130703 31818 log.cc:824] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:10.131909 31817 raft_consensus.cc:348] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } }
I20260619 18:51:10.132200 31817 raft_consensus.cc:374] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:10.132294 31817 raft_consensus.cc:729] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b8ae07b0f7b4b1581c1a5c25cea90ff, State: Initialized, Role: FOLLOWER
I20260619 18:51:10.132318 31819 raft_consensus.cc:348] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } }
I20260619 18:51:10.132630 31819 raft_consensus.cc:374] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:10.132695 31819 raft_consensus.cc:729] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 662ffc8af1bf40f9af94356c13c44cec, State: Initialized, Role: FOLLOWER
I20260619 18:51:10.132880 31818 tablet_bootstrap.cc:492] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef: No bootstrap required, opened a new log
I20260619 18:51:10.132797 31817 consensus_queue.cc:260] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } }
I20260619 18:51:10.133137 31818 ts_tablet_manager.cc:1397] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:51:10.133122 31819 consensus_queue.cc:260] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } }
I20260619 18:51:10.133517 31800 heartbeater.cc:503] Master 127.12.234.254:36233 was elected leader, sending a full tablet report...
I20260619 18:51:10.133795 31666 heartbeater.cc:503] Master 127.12.234.254:36233 was elected leader, sending a full tablet report...
I20260619 18:51:10.133875 31817 ts_tablet_manager.cc:1428] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 18:51:10.134150 31819 ts_tablet_manager.cc:1428] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260619 18:51:10.138621 31818 raft_consensus.cc:348] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } }
I20260619 18:51:10.139160 31818 raft_consensus.cc:374] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:10.139226 31818 raft_consensus.cc:729] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6caadb549a7d4e109fe12f3e9c5bd3ef, State: Initialized, Role: FOLLOWER
I20260619 18:51:10.139637 31818 consensus_queue.cc:260] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } }
I20260619 18:51:10.140280 31532 heartbeater.cc:503] Master 127.12.234.254:36233 was elected leader, sending a full tablet report...
I20260619 18:51:10.140578 31818 ts_tablet_manager.cc:1428] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 18:51:10.272495 31823 raft_consensus.cc:482] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:51:10.272743 31823 raft_consensus.cc:504] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } }
I20260619 18:51:10.274036 31823 leader_election.cc:288] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6caadb549a7d4e109fe12f3e9c5bd3ef (127.12.234.193:35463), 662ffc8af1bf40f9af94356c13c44cec (127.12.234.194:37217)
I20260619 18:51:10.279042 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.274427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:34513 (local address 127.12.234.194:37217)
0619 18:51:10.274582 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.274587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.274942 (+ 355us) server_negotiation.cc:407] Connection header received
0619 18:51:10.275170 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.275174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.275227 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.275296 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:10.275537 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.276037 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.277784 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.278458 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.278579 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.278632 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.278818 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.278885 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 18:51:10.279140 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.274558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:37217 (local address 127.12.234.195:34513)
0619 18:51:10.274860 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:10.274873 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:10.275042 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:10.275311 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:10.275314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:10.275329 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:10.275428 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.275433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.276143 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.276146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.277647 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.277659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.278580 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.278584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.278686 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.278714 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 18:51:10.278806 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 18:51:10.278884 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":113,"spinlock_wait_cycles":28288,"thread_start_us":119,"threads_started":1}
I20260619 18:51:10.279582 31802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.274330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:35463 (local address 127.12.234.195:35089)
0619 18:51:10.274527 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:10.274542 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:10.274636 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:10.274945 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:10.274949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:10.274968 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:10.275101 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.275108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.276164 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.276167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.277652 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.277659 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.279148 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.279151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.279264 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.279285 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 18:51:10.279380 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 18:51:10.279465 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":111,"spinlock_wait_cycles":1792}
I20260619 18:51:10.279623 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.274432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:35089 (local address 127.12.234.193:35463)
0619 18:51:10.274694 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.274698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.274714 (+ 16us) server_negotiation.cc:407] Connection header received
0619 18:51:10.274771 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.274774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.274836 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.274918 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:10.275238 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.276041 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.278274 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.278839 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.278992 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.279069 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.279416 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.279497 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260619 18:51:10.280370 31486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4295675b15a4ebea5bea8a84b629bb3" candidate_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" is_pre_election: true
I20260619 18:51:10.280371 31620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4295675b15a4ebea5bea8a84b629bb3" candidate_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "662ffc8af1bf40f9af94356c13c44cec" is_pre_election: true
I20260619 18:51:10.280689 31620 raft_consensus.cc:2454] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b8ae07b0f7b4b1581c1a5c25cea90ff in term 0.
I20260619 18:51:10.280689 31486 raft_consensus.cc:2454] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b8ae07b0f7b4b1581c1a5c25cea90ff in term 0.
I20260619 18:51:10.281205 31690 leader_election.cc:302] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 662ffc8af1bf40f9af94356c13c44cec, 6caadb549a7d4e109fe12f3e9c5bd3ef, 7b8ae07b0f7b4b1581c1a5c25cea90ff; no voters:
I20260619 18:51:10.281500 31823 raft_consensus.cc:2781] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:51:10.281581 31823 raft_consensus.cc:482] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:51:10.281620 31823 raft_consensus.cc:3037] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:10.282743 31823 raft_consensus.cc:504] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } }
I20260619 18:51:10.283138 31823 leader_election.cc:288] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [CANDIDATE]: Term 1 election: Requested vote from peers 6caadb549a7d4e109fe12f3e9c5bd3ef (127.12.234.193:35463), 662ffc8af1bf40f9af94356c13c44cec (127.12.234.194:37217)
I20260619 18:51:10.283588 31486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4295675b15a4ebea5bea8a84b629bb3" candidate_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef"
I20260619 18:51:10.283730 31486 raft_consensus.cc:3037] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:10.285075 31486 raft_consensus.cc:2454] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b8ae07b0f7b4b1581c1a5c25cea90ff in term 1.
I20260619 18:51:10.285380 31620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4295675b15a4ebea5bea8a84b629bb3" candidate_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "662ffc8af1bf40f9af94356c13c44cec"
I20260619 18:51:10.285461 31690 leader_election.cc:302] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6caadb549a7d4e109fe12f3e9c5bd3ef, 7b8ae07b0f7b4b1581c1a5c25cea90ff; no voters:
I20260619 18:51:10.285514 31620 raft_consensus.cc:3037] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:10.285645 31823 raft_consensus.cc:2781] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:10.285880 31823 raft_consensus.cc:686] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 1 LEADER]: Becoming Leader. State: Replica: 7b8ae07b0f7b4b1581c1a5c25cea90ff, State: Running, Role: LEADER
I20260619 18:51:10.286267 31823 consensus_queue.cc:237] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } }
I20260619 18:51:10.286837 31620 raft_consensus.cc:2454] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b8ae07b0f7b4b1581c1a5c25cea90ff in term 1.
I20260619 18:51:10.289705 31338 catalog_manager.cc:5495] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b8ae07b0f7b4b1581c1a5c25cea90ff (127.12.234.195). New cstate: current_term: 1 leader_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8ae07b0f7b4b1581c1a5c25cea90ff" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 33667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 } health_report { overall_health: UNKNOWN } } }
W20260619 18:51:10.299309 31801 tablet.cc:2369] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:10.304956 31816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.301369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.12.234.195:33667)
0619 18:51:10.301506 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.301510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.301538 (+ 28us) server_negotiation.cc:407] Connection header received
0619 18:51:10.301639 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.301641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.301696 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.301783 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:10.302094 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.302657 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.303515 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.303737 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.303830 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.304142 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:10.304594 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:10.304706 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.304775 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.304814 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 18:51:10.311205 31486 raft_consensus.cc:1261] T c4295675b15a4ebea5bea8a84b629bb3 P 6caadb549a7d4e109fe12f3e9c5bd3ef [term 1 FOLLOWER]: Refusing update from remote peer 7b8ae07b0f7b4b1581c1a5c25cea90ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:10.311205 31620 raft_consensus.cc:1261] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [term 1 FOLLOWER]: Refusing update from remote peer 7b8ae07b0f7b4b1581c1a5c25cea90ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:10.311918 31823 consensus_queue.cc:1035] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "6caadb549a7d4e109fe12f3e9c5bd3ef" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 35463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:10.312165 31827 consensus_queue.cc:1035] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "662ffc8af1bf40f9af94356c13c44cec" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 37217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:10.320336 31832 mvcc.cc:205] Tried to move back new op lower bound from 7298642207986712576 to 7298642207895646208. Current Snapshot: MvccSnapshot[applied={T|T < 7298642207986712576}]
I20260619 18:51:10.320340 31831 mvcc.cc:205] Tried to move back new op lower bound from 7298642207986712576 to 7298642207895646208. Current Snapshot: MvccSnapshot[applied={T|T < 7298642207986712576}]
I20260619 18:51:10.330343 31341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:10.330526 31341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:10.333236 31341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc560a59204a47feb65c897fc3457dd1: Sending DeleteTablet for 3 replicas of tablet c4295675b15a4ebea5bea8a84b629bb3
I20260619 18:51:10.334237 31466 tablet_service.cc:1514] Processing DeleteTablet for tablet c4295675b15a4ebea5bea8a84b629bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:10 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:46136
I20260619 18:51:10.334439 31734 tablet_service.cc:1514] Processing DeleteTablet for tablet c4295675b15a4ebea5bea8a84b629bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:10 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:43358
I20260619 18:51:10.334838 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 31403
I20260619 18:51:10.334972 31600 tablet_service.cc:1514] Processing DeleteTablet for tablet c4295675b15a4ebea5bea8a84b629bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:10 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:42650
I20260619 18:51:10.335687 31842 tablet_replica.cc:330] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: stopping tablet replica
I20260619 18:51:10.336059 31842 raft_consensus.cc:2229] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:51:10.336408 31842 raft_consensus.cc:2258] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:10.338171 31842 ts_tablet_manager.cc:1905] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:10.338482 31841 tablet_replica.cc:330] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: stopping tablet replica
I20260619 18:51:10.338814 31841 raft_consensus.cc:2229] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:51:10.339260 31841 raft_consensus.cc:2258] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:10.340934 31841 ts_tablet_manager.cc:1905] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:10.341359 31842 ts_tablet_manager.cc:1918] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:10.341466 31842 log.cc:1196] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/c4295675b15a4ebea5bea8a84b629bb3
I20260619 18:51:10.341771 31842 ts_tablet_manager.cc:1939] T c4295675b15a4ebea5bea8a84b629bb3 P 662ffc8af1bf40f9af94356c13c44cec: Deleting consensus metadata
I20260619 18:51:10.343294 31326 catalog_manager.cc:4841] TS 662ffc8af1bf40f9af94356c13c44cec (127.12.234.194:37217): tablet c4295675b15a4ebea5bea8a84b629bb3 (table test-table [id=edca2f9defae4a5ea8c2c319f079e037]) successfully deleted
W20260619 18:51:10.343787 31328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:35463 (error 108)
I20260619 18:51:10.344094 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 31537
I20260619 18:51:10.343890 31841 ts_tablet_manager.cc:1918] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:10.344846 31841 log.cc:1196] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/c4295675b15a4ebea5bea8a84b629bb3
I20260619 18:51:10.345104 31841 ts_tablet_manager.cc:1939] T c4295675b15a4ebea5bea8a84b629bb3 P 7b8ae07b0f7b4b1581c1a5c25cea90ff: Deleting consensus metadata
I20260619 18:51:10.346120 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.345802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:35463 (local address 127.0.0.1:46148)
0619 18:51:10.345951 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:10.346028 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:35463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 18:51:10.346378 31326 catalog_manager.cc:4841] TS 7b8ae07b0f7b4b1581c1a5c25cea90ff (127.12.234.195:33667): tablet c4295675b15a4ebea5bea8a84b629bb3 (table test-table [id=edca2f9defae4a5ea8c2c319f079e037]) successfully deleted
W20260619 18:51:10.346405 31328 catalog_manager.cc:4584] TS 6caadb549a7d4e109fe12f3e9c5bd3ef (127.12.234.193:35463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c4295675b15a4ebea5bea8a84b629bb3: Network error: Client connection negotiation failed: client connection to 127.12.234.193:35463: connect: Connection refused (error 111)
I20260619 18:51:10.349912 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 31671
I20260619 18:51:10.355105 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 31309
2026-06-19T18:51:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2337 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T18:51:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:10Z Disabled control of system clock
I20260619 18:51:10.384490 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:38807
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:38807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 18:51:10.498821 31847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:10.499181 31847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:10.499230 31847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:10.503460 31847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:10.503535 31847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:10.503551 31847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:10.503566 31847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:10.503582 31847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:10.508594 31847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:38807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:38807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:10.509809 31847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:10.511245 31847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:10.520251 31852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:10.520252 31855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:10.520238 31853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:10.520493 31847 server_base.cc:1034] running on GCE node
I20260619 18:51:10.521078 31847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:10.522296 31847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:10.523475 31847 hybrid_clock.cc:648] HybridClock initialized: now 1781895070523459 us; error 34 us; skew 500 ppm
I20260619 18:51:10.525492 31847 webserver.cc:460] Webserver started at http://127.12.234.254:33717/ using document root <none> and password file <none>
I20260619 18:51:10.526079 31847 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:10.526152 31847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:10.526409 31847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:10.528568 31847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "539ca873880d476fa3788ab5339978bc"
format_stamp: "Formatted at 2026-06-19 18:51:10 on dist-test-slave-fljv"
server_key: "9f8ea344e36150b386e9ac6798a88f85"
server_key_iv: "ab97878b7cdc28109fa0842380851acc"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.529170 31847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "539ca873880d476fa3788ab5339978bc"
format_stamp: "Formatted at 2026-06-19 18:51:10 on dist-test-slave-fljv"
server_key: "9f8ea344e36150b386e9ac6798a88f85"
server_key_iv: "ab97878b7cdc28109fa0842380851acc"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.533192 31847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260619 18:51:10.535971 31861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:10.537533 31847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:51:10.537690 31847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "539ca873880d476fa3788ab5339978bc"
format_stamp: "Formatted at 2026-06-19 18:51:10 on dist-test-slave-fljv"
server_key: "9f8ea344e36150b386e9ac6798a88f85"
server_key_iv: "ab97878b7cdc28109fa0842380851acc"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.537819 31847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:10.554350 31847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:10.555124 31847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:10.555316 31847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:10.562012 31847 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:38807
I20260619 18:51:10.562037 31913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:38807 every 8 connection(s)
I20260619 18:51:10.563058 31847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:10.566555 31914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:10.570505 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 31847
I20260619 18:51:10.570658 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:10.570977 13227 external_mini_cluster.cc:1428] Setting key b5a4896ec94b7a99acc3864db282a5af
I20260619 18:51:10.573175 31914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc: Bootstrap starting.
I20260619 18:51:10.575551 31914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:10.576323 31914 log.cc:824] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:10.578370 31914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc: No bootstrap required, opened a new log
I20260619 18:51:10.580971 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.572693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.12.234.254:38807)
0619 18:51:10.573475 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.573494 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.573542 (+ 48us) server_negotiation.cc:407] Connection header received
0619 18:51:10.574295 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.574317 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.574768 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.575114 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:10.575582 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.576671 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.577597 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.578217 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.578370 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.579190 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:10.579195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:10.579209 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:10.579613 (+ 404us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:10.579671 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:10.579692 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:10.579786 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.579922 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.580211 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":144,"threads_started":1}
I20260619 18:51:10.582154 31914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539ca873880d476fa3788ab5339978bc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38807 } }
I20260619 18:51:10.582437 31914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:10.582497 31914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 539ca873880d476fa3788ab5339978bc, State: Initialized, Role: FOLLOWER
I20260619 18:51:10.582937 31914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539ca873880d476fa3788ab5339978bc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38807 } }
I20260619 18:51:10.583071 31914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:10.583137 31914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:10.583235 31914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:10.584234 31914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539ca873880d476fa3788ab5339978bc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38807 } }
I20260619 18:51:10.584622 31914 leader_election.cc:302] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 539ca873880d476fa3788ab5339978bc; no voters:
I20260619 18:51:10.584978 31914 leader_election.cc:288] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:10.585080 31919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:10.585242 31919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [term 1 LEADER]: Becoming Leader. State: Replica: 539ca873880d476fa3788ab5339978bc, State: Running, Role: LEADER
I20260619 18:51:10.585553 31919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539ca873880d476fa3788ab5339978bc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38807 } }
I20260619 18:51:10.586373 31914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:10.587385 31921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 539ca873880d476fa3788ab5339978bc. Latest consensus state: current_term: 1 leader_uuid: "539ca873880d476fa3788ab5339978bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539ca873880d476fa3788ab5339978bc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38807 } } }
I20260619 18:51:10.587363 31920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "539ca873880d476fa3788ab5339978bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539ca873880d476fa3788ab5339978bc" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38807 } } }
I20260619 18:51:10.587544 31921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:10.587549 31920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:10.588157 31928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:10.591498 31928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:10.597541 31928 catalog_manager.cc:1347] Generated new cluster ID: f939a28a92cb47b7bad47d0271c60b75
I20260619 18:51:10.597640 31928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:10.611582 31928 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:10.612574 31928 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:10.618875 31928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc: Generated new TSK 0
I20260619 18:51:10.619889 31928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:10.633623 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:38807
--builtin_ntp_servers=127.12.234.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 18:51:10.746860 31938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:10.747162 31938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:10.747224 31938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:10.751389 31938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:10.751453 31938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:10.751533 31938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:51:10.756399 31938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:38807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:10.757514 31938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:10.758939 31938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:10.768803 31946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:10.768817 31943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:10.769088 31938 server_base.cc:1034] running on GCE node
W20260619 18:51:10.769223 31944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:10.769666 31938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:10.770382 31938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:10.771559 31938 hybrid_clock.cc:648] HybridClock initialized: now 1781895070771535 us; error 36 us; skew 500 ppm
I20260619 18:51:10.773447 31938 webserver.cc:460] Webserver started at http://127.12.234.193:36635/ using document root <none> and password file <none>
I20260619 18:51:10.774032 31938 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:10.774099 31938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:10.774333 31938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:10.776417 31938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d"
format_stamp: "Formatted at 2026-06-19 18:51:10 on dist-test-slave-fljv"
server_key: "44403e86509eaca5f8b0e932b1569f32"
server_key_iv: "d4fc81493a0b505f4b285fd476549564"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.776965 31938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d"
format_stamp: "Formatted at 2026-06-19 18:51:10 on dist-test-slave-fljv"
server_key: "44403e86509eaca5f8b0e932b1569f32"
server_key_iv: "d4fc81493a0b505f4b285fd476549564"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.780668 31938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 18:51:10.783246 31952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:10.784536 31938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:51:10.784649 31938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d"
format_stamp: "Formatted at 2026-06-19 18:51:10 on dist-test-slave-fljv"
server_key: "44403e86509eaca5f8b0e932b1569f32"
server_key_iv: "d4fc81493a0b505f4b285fd476549564"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.784757 31938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:10.803210 31938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:10.803951 31938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:10.804132 31938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:10.804756 31938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:10.805725 31938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:10.805771 31938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:10.805823 31938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:10.805850 31938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:10.814555 31938 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:44857
I20260619 18:51:10.814600 32065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:44857 every 8 connection(s)
I20260619 18:51:10.815534 31938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:51:10.820451 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 31938
I20260619 18:51:10.820600 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:51:10.820935 13227 external_mini_cluster.cc:1428] Setting key 6e6a14ac7ab4868fd29ac3189b7cb518
I20260619 18:51:10.823335 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:38807
--builtin_ntp_servers=127.12.234.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 18:51:10.826685 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.817535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:58509 (local address 127.12.234.254:38807)
0619 18:51:10.817665 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:10.817669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:10.818918 (+ 1249us) server_negotiation.cc:407] Connection header received
0619 18:51:10.820407 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:10.820411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:10.820474 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:10.820561 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:10.822071 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.822766 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.824524 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.824860 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.824982 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.826216 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:10.826220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:10.826222 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:10.826238 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:10.826261 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:10.826271 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:10.826337 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:10.826472 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 18:51:10.826538 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:51:10.827672 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:10.817963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:38807 (local address 127.12.234.193:58509)
0619 18:51:10.818745 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:10.818790 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:10.820154 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:10.820856 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:10.820868 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:10.821103 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:10.821812 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.821834 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.822905 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.822909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.824362 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:10.824376 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:10.825035 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:10.825038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:10.825206 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:10.825852 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:51:10.825853 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:10.825898 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:51:10.826024 (+ 126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:10.826348 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:10.826354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:10.826359 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 18:51:10.826458 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 18:51:10.826720 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":148,"threads_started":1}
I20260619 18:51:10.829090 32066 heartbeater.cc:347] Connected to a master server at 127.12.234.254:38807
I20260619 18:51:10.829342 32066 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:10.829902 32066 heartbeater.cc:511] Master 127.12.234.254:38807 requested a full tablet report, sending...
I20260619 18:51:10.831463 31878 ts_manager.cc:194] Registered new tserver with Master: 77d1cb5e84ca4d62a8b4dc58c64b9a7d (127.12.234.193:44857)
I20260619 18:51:10.832670 31878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.193:58509
W20260619 18:51:10.936017 32069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:10.936304 32069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:10.936343 32069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:10.940464 32069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:10.940541 32069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:10.940604 32069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:51:10.945734 32069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:38807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:10.946879 32069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:10.948241 32069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:10.958153 32074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:10.958230 32077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:10.958251 32075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:10.958303 32069 server_base.cc:1034] running on GCE node
I20260619 18:51:10.958807 32069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:10.959511 32069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:10.960677 32069 hybrid_clock.cc:648] HybridClock initialized: now 1781895070960645 us; error 45 us; skew 500 ppm
I20260619 18:51:10.962636 32069 webserver.cc:460] Webserver started at http://127.12.234.194:41719/ using document root <none> and password file <none>
I20260619 18:51:10.963201 32069 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:10.963261 32069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:10.963425 32069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:10.965505 32069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "ac0b56b006514a4495e128b32f0493c9"
format_stamp: "Formatted at 2026-06-19 18:51:10 on dist-test-slave-fljv"
server_key: "fbc843000734fd5fb101ef4b11e42b20"
server_key_iv: "54ae1692dd93c3cc1c989673ccc62595"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.966038 32069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "ac0b56b006514a4495e128b32f0493c9"
format_stamp: "Formatted at 2026-06-19 18:51:10 on dist-test-slave-fljv"
server_key: "fbc843000734fd5fb101ef4b11e42b20"
server_key_iv: "54ae1692dd93c3cc1c989673ccc62595"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.969923 32069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 18:51:10.972711 32083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:10.974213 32069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:10.974334 32069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "ac0b56b006514a4495e128b32f0493c9"
format_stamp: "Formatted at 2026-06-19 18:51:10 on dist-test-slave-fljv"
server_key: "fbc843000734fd5fb101ef4b11e42b20"
server_key_iv: "54ae1692dd93c3cc1c989673ccc62595"
server_key_version: "encryptionkey@0"
I20260619 18:51:10.974442 32069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:10.990268 32069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:10.990978 32069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:10.991168 32069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:10.991777 32069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:10.992750 32069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:10.992799 32069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:10.992851 32069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:10.992872 32069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:11.001588 32069 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:35387
I20260619 18:51:11.001617 32196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:35387 every 8 connection(s)
I20260619 18:51:11.002589 32069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:51:11.010334 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 32069
I20260619 18:51:11.010468 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:51:11.010808 13227 external_mini_cluster.cc:1428] Setting key d1e2692a2d1ed7759b2bc5613bce010a
I20260619 18:51:11.013127 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.004578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:58563 (local address 127.12.234.254:38807)
0619 18:51:11.004728 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:11.004734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:11.005978 (+ 1244us) server_negotiation.cc:407] Connection header received
0619 18:51:11.007451 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:11.007455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:11.007524 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:11.007618 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:11.008731 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.009346 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.010936 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.011219 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.011315 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.012577 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:11.012581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:11.012584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:11.012598 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:11.012622 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:11.012640 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:11.012720 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:11.012912 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 18:51:11.012981 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 18:51:11.013113 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:38807
--builtin_ntp_servers=127.12.234.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 18:51:11.014111 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.005043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:38807 (local address 127.12.234.194:58563)
0619 18:51:11.005794 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:11.005840 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:11.007210 (+ 1370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:11.007819 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:11.007826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:11.007927 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:11.008536 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.008550 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.009508 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.009510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.010792 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.010805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.011386 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.011390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.011560 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.012247 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:51:11.012250 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:11.012312 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:51:11.012403 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:11.012752 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:11.012761 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:11.012766 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 18:51:11.012897 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 18:51:11.013182 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":119,"threads_started":1}
I20260619 18:51:11.015504 32197 heartbeater.cc:347] Connected to a master server at 127.12.234.254:38807
I20260619 18:51:11.015789 32197 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:11.016537 32197 heartbeater.cc:511] Master 127.12.234.254:38807 requested a full tablet report, sending...
I20260619 18:51:11.017853 31878 ts_manager.cc:194] Registered new tserver with Master: ac0b56b006514a4495e128b32f0493c9 (127.12.234.194:35387)
I20260619 18:51:11.018594 31878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.194:58563
W20260619 18:51:11.128808 32200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:11.129115 32200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:11.129156 32200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:11.133392 32200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:11.133471 32200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:11.133528 32200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:51:11.138490 32200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:38807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:11.139667 32200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:11.141089 32200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:51:11.151473 32200 server_base.cc:1034] running on GCE node
W20260619 18:51:11.151445 32206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:11.151501 32208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:11.151664 32205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:11.152079 32200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:11.152751 32200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:11.153914 32200 hybrid_clock.cc:648] HybridClock initialized: now 1781895071153881 us; error 44 us; skew 500 ppm
I20260619 18:51:11.155788 32200 webserver.cc:460] Webserver started at http://127.12.234.195:43989/ using document root <none> and password file <none>
I20260619 18:51:11.156386 32200 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:11.156462 32200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:11.156675 32200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:11.158902 32200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "04c61da5761349078acc61eb62bf1f01"
format_stamp: "Formatted at 2026-06-19 18:51:11 on dist-test-slave-fljv"
server_key: "835a10da37d1c1712e32cad6329ea34a"
server_key_iv: "8cd31f7d9adc6f119551541dbb398187"
server_key_version: "encryptionkey@0"
I20260619 18:51:11.159505 32200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "04c61da5761349078acc61eb62bf1f01"
format_stamp: "Formatted at 2026-06-19 18:51:11 on dist-test-slave-fljv"
server_key: "835a10da37d1c1712e32cad6329ea34a"
server_key_iv: "8cd31f7d9adc6f119551541dbb398187"
server_key_version: "encryptionkey@0"
I20260619 18:51:11.163371 32200 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:11.166108 32214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:11.167485 32200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:11.167604 32200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "04c61da5761349078acc61eb62bf1f01"
format_stamp: "Formatted at 2026-06-19 18:51:11 on dist-test-slave-fljv"
server_key: "835a10da37d1c1712e32cad6329ea34a"
server_key_iv: "8cd31f7d9adc6f119551541dbb398187"
server_key_version: "encryptionkey@0"
I20260619 18:51:11.167722 32200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:11.186012 32200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:11.186784 32200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:11.186954 32200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:11.187572 32200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:11.188581 32200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:11.188625 32200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:11.188683 32200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:11.188712 32200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:11.196877 32200 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:39547
I20260619 18:51:11.196902 32327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:39547 every 8 connection(s)
I20260619 18:51:11.197849 32200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:51:11.200161 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 32200
I20260619 18:51:11.200259 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:51:11.200532 13227 external_mini_cluster.cc:1428] Setting key a9703af01dfbeb5b0418e0fc18b48960
I20260619 18:51:11.207086 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.199687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:42721 (local address 127.12.234.254:38807)
0619 18:51:11.199856 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:11.199861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:11.201034 (+ 1173us) server_negotiation.cc:407] Connection header received
0619 18:51:11.202631 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:11.202634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:11.202690 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:11.202756 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:11.203656 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.204199 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.205281 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.205497 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.205574 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.206647 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:11.206651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:11.206653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:11.206665 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:11.206683 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:11.206691 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:11.206744 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:11.206887 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 18:51:11.206937 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 18:51:11.207942 32330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.200146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:38807 (local address 127.12.234.195:42721)
0619 18:51:11.200879 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:11.200930 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:11.202410 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:11.202943 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:11.202951 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:11.203042 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:11.203534 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.203546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.204289 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.204291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.205182 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.205191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.205639 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.205641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.205781 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.206385 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:51:11.206387 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:11.206440 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:51:11.206511 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:11.206758 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:11.206764 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:11.206769 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 18:51:11.206886 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 18:51:11.207134 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":130,"threads_started":1}
I20260619 18:51:11.209085 32328 heartbeater.cc:347] Connected to a master server at 127.12.234.254:38807
I20260619 18:51:11.209317 32328 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:11.209836 32328 heartbeater.cc:511] Master 127.12.234.254:38807 requested a full tablet report, sending...
I20260619 18:51:11.210851 31878 ts_manager.cc:194] Registered new tserver with Master: 04c61da5761349078acc61eb62bf1f01 (127.12.234.195:39547)
I20260619 18:51:11.211433 31878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.195:42721
I20260619 18:51:11.214500 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:51:11.220387 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.216452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.12.234.254:38807)
0619 18:51:11.216594 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:11.216599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:11.216714 (+ 115us) server_negotiation.cc:407] Connection header received
0619 18:51:11.216816 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:11.216818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:11.216857 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:11.216922 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:11.217238 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.217758 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.218655 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.218869 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.218946 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.219779 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:11.219782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:11.219784 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:11.219797 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:11.219817 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:11.219825 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:11.219939 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:11.220186 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 18:51:11.220233 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 18:51:11.223788 31878 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:51:11.226953 31878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:51:11.242098 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.237074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39082 (local address 127.12.234.195:39547)
0619 18:51:11.237480 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:11.237496 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:11.237518 (+ 22us) server_negotiation.cc:407] Connection header received
0619 18:51:11.237620 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:11.237625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:11.237806 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:11.238007 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:11.238251 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.239288 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.240796 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.241384 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.241494 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.241763 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:11.241814 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 18:51:11.241915 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":134,"threads_started":1}
I20260619 18:51:11.242142 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.236055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:44857 (local address 127.0.0.1:43300)
0619 18:51:11.236748 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:11.236760 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:11.236844 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:11.238103 (+ 1259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:11.238106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:11.238120 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:11.238210 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.238216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.239298 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.239302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.240493 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.240503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.241599 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.241603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.241688 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.241711 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 18:51:11.241770 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 18:51:11.241946 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":82,"thread_start_us":104,"threads_started":1}
I20260619 18:51:11.242372 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.235968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:35387 (local address 127.0.0.1:46518)
0619 18:51:11.236406 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:11.236454 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:11.236603 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:11.237392 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:11.237398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:11.237421 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:11.237566 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.237573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.239199 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.239202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.240655 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.240667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.241893 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.241897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.241987 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.242008 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 18:51:11.242069 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 18:51:11.242132 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":80,"thread_start_us":74,"threads_started":1}
I20260619 18:51:11.242506 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.236095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.12.234.194:35387)
0619 18:51:11.236466 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:11.236477 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:11.236744 (+ 267us) server_negotiation.cc:407] Connection header received
0619 18:51:11.236827 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:11.236832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:11.237065 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:11.237226 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:11.237802 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.239077 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.240778 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.241512 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.241664 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.241835 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:11.242251 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 18:51:11.242359 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":85,"threads_started":1}
I20260619 18:51:11.242585 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.236710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:39547 (local address 127.0.0.1:39082)
0619 18:51:11.237209 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:11.237221 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:11.237301 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:11.237993 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:11.237996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:11.238024 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:11.238122 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.238129 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.239370 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.239373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.240685 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.240692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.241493 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.241497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.241643 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.241686 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 18:51:11.241788 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 18:51:11.241938 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":86,"thread_start_us":79,"threads_started":1}
I20260619 18:51:11.245172 32262 tablet_service.cc:1467] Processing CreateTablet for tablet ce05bf0e86954bea9810900c42780f41 (DEFAULT_TABLE table=test-table [id=5eb5b2ff37d64412a04e026bfb34d1e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:11.245167 32131 tablet_service.cc:1467] Processing CreateTablet for tablet ce05bf0e86954bea9810900c42780f41 (DEFAULT_TABLE table=test-table [id=5eb5b2ff37d64412a04e026bfb34d1e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:11.246073 32262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce05bf0e86954bea9810900c42780f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:11.246073 32131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce05bf0e86954bea9810900c42780f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:11.248059 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.236724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.12.234.193:44857)
0619 18:51:11.237456 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:11.237470 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:11.237493 (+ 23us) server_negotiation.cc:407] Connection header received
0619 18:51:11.237705 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:11.237709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:11.237890 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:11.238328 (+ 438us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:11.238436 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.239183 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.240623 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.241441 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.242104 (+ 663us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.247717 (+ 5613us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:11.247779 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 18:51:11.247893 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":140,"threads_started":1}
I20260619 18:51:11.251207 32000 tablet_service.cc:1467] Processing CreateTablet for tablet ce05bf0e86954bea9810900c42780f41 (DEFAULT_TABLE table=test-table [id=5eb5b2ff37d64412a04e026bfb34d1e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:11.252396 32345 tablet_bootstrap.cc:492] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: Bootstrap starting.
I20260619 18:51:11.253381 32000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce05bf0e86954bea9810900c42780f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:11.254541 32346 tablet_bootstrap.cc:492] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: Bootstrap starting.
I20260619 18:51:11.254832 32345 tablet_bootstrap.cc:654] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:11.255657 32345 log.cc:824] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:11.256783 32346 tablet_bootstrap.cc:654] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:11.257457 32345 tablet_bootstrap.cc:492] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: No bootstrap required, opened a new log
I20260619 18:51:11.257665 32345 ts_tablet_manager.cc:1397] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 18:51:11.257660 32346 log.cc:824] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:11.260236 32346 tablet_bootstrap.cc:492] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: No bootstrap required, opened a new log
I20260619 18:51:11.260447 32346 ts_tablet_manager.cc:1397] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:51:11.261173 32349 tablet_bootstrap.cc:492] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d: Bootstrap starting.
I20260619 18:51:11.261747 32345 raft_consensus.cc:348] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } }
I20260619 18:51:11.262049 32345 raft_consensus.cc:374] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:11.262116 32345 raft_consensus.cc:729] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c61da5761349078acc61eb62bf1f01, State: Initialized, Role: FOLLOWER
I20260619 18:51:11.262615 32345 consensus_queue.cc:260] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } }
I20260619 18:51:11.263340 32328 heartbeater.cc:503] Master 127.12.234.254:38807 was elected leader, sending a full tablet report...
I20260619 18:51:11.263669 32345 ts_tablet_manager.cc:1428] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:51:11.263711 32349 tablet_bootstrap.cc:654] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:11.264523 32349 log.cc:824] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:11.264386 32346 raft_consensus.cc:348] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } }
I20260619 18:51:11.264623 32346 raft_consensus.cc:374] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:11.264680 32346 raft_consensus.cc:729] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac0b56b006514a4495e128b32f0493c9, State: Initialized, Role: FOLLOWER
I20260619 18:51:11.265143 32346 consensus_queue.cc:260] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } }
I20260619 18:51:11.265821 32197 heartbeater.cc:503] Master 127.12.234.254:38807 was elected leader, sending a full tablet report...
I20260619 18:51:11.267127 32349 tablet_bootstrap.cc:492] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d: No bootstrap required, opened a new log
I20260619 18:51:11.267326 32349 ts_tablet_manager.cc:1397] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:51:11.267361 32346 ts_tablet_manager.cc:1428] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260619 18:51:11.270706 32349 raft_consensus.cc:348] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } }
I20260619 18:51:11.270928 32349 raft_consensus.cc:374] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:11.270959 32349 raft_consensus.cc:729] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d1cb5e84ca4d62a8b4dc58c64b9a7d, State: Initialized, Role: FOLLOWER
I20260619 18:51:11.271348 32349 consensus_queue.cc:260] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } }
I20260619 18:51:11.272174 32066 heartbeater.cc:503] Master 127.12.234.254:38807 was elected leader, sending a full tablet report...
I20260619 18:51:11.272218 32349 ts_tablet_manager.cc:1428] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260619 18:51:11.316689 32067 tablet.cc:2369] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:51:11.448802 32329 tablet.cc:2369] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 18:51:11.503839 32198 tablet.cc:2369] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:11.576323 32353 raft_consensus.cc:482] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:51:11.576545 32353 raft_consensus.cc:504] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } }
I20260619 18:51:11.577816 32353 leader_election.cc:288] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac0b56b006514a4495e128b32f0493c9 (127.12.234.194:35387), 04c61da5761349078acc61eb62bf1f01 (127.12.234.195:39547)
I20260619 18:51:11.582893 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.578156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:37909 (local address 127.12.234.194:35387)
0619 18:51:11.578304 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:11.578309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:11.578401 (+ 92us) server_negotiation.cc:407] Connection header received
0619 18:51:11.578632 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:11.578635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:11.578689 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:11.578733 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:11.579063 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.579876 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.581713 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.582311 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.582411 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.582449 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:11.582688 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 18:51:11.582758 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 18:51:11.582893 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.578609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:37241 (local address 127.12.234.195:39547)
0619 18:51:11.578723 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:11.578727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:11.578743 (+ 16us) server_negotiation.cc:407] Connection header received
0619 18:51:11.578886 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:11.578889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:11.578932 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:11.579002 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:11.579245 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.579945 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.581638 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.582264 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.582390 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.582441 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:11.582688 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 18:51:11.582758 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 18:51:11.582953 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.578444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:39547 (local address 127.12.234.193:37241)
0619 18:51:11.578682 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:11.578695 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:11.578789 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:11.579024 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:11.579035 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:11.579056 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:11.579152 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.579156 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.580085 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.580088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.581489 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.581501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.582384 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.582387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.582531 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.582593 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 18:51:11.582667 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 18:51:11.582761 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":114,"thread_start_us":71,"threads_started":1}
I20260619 18:51:11.583323 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.578051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:35387 (local address 127.12.234.193:37909)
0619 18:51:11.578311 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:11.578326 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:11.578448 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:11.578853 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:11.578855 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:11.578870 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:11.578965 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.578969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.579989 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.579993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.581489 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:11.581500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.582412 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.582415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:11.582511 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.582590 (+ 79us) client_negotiation.cc:769] Sending connection context
0619 18:51:11.582667 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 18:51:11.582761 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":166,"thread_start_us":113,"threads_started":1}
I20260619 18:51:11.583698 32282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce05bf0e86954bea9810900c42780f41" candidate_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c61da5761349078acc61eb62bf1f01" is_pre_election: true
I20260619 18:51:11.583886 32151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce05bf0e86954bea9810900c42780f41" candidate_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac0b56b006514a4495e128b32f0493c9" is_pre_election: true
I20260619 18:51:11.584045 32282 raft_consensus.cc:2454] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77d1cb5e84ca4d62a8b4dc58c64b9a7d in term 0.
I20260619 18:51:11.584131 32151 raft_consensus.cc:2454] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77d1cb5e84ca4d62a8b4dc58c64b9a7d in term 0.
I20260619 18:51:11.584560 31955 leader_election.cc:302] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c61da5761349078acc61eb62bf1f01, 77d1cb5e84ca4d62a8b4dc58c64b9a7d; no voters:
I20260619 18:51:11.584816 32353 raft_consensus.cc:2781] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:51:11.584901 32353 raft_consensus.cc:482] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:51:11.584931 32353 raft_consensus.cc:3037] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:11.586035 32353 raft_consensus.cc:504] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } }
I20260619 18:51:11.586405 32353 leader_election.cc:288] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [CANDIDATE]: Term 1 election: Requested vote from peers ac0b56b006514a4495e128b32f0493c9 (127.12.234.194:35387), 04c61da5761349078acc61eb62bf1f01 (127.12.234.195:39547)
I20260619 18:51:11.586812 32151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce05bf0e86954bea9810900c42780f41" candidate_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac0b56b006514a4495e128b32f0493c9"
I20260619 18:51:11.586864 32282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce05bf0e86954bea9810900c42780f41" candidate_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04c61da5761349078acc61eb62bf1f01"
I20260619 18:51:11.586961 32151 raft_consensus.cc:3037] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:11.586985 32282 raft_consensus.cc:3037] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:11.588289 32282 raft_consensus.cc:2454] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77d1cb5e84ca4d62a8b4dc58c64b9a7d in term 1.
I20260619 18:51:11.588308 32151 raft_consensus.cc:2454] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77d1cb5e84ca4d62a8b4dc58c64b9a7d in term 1.
I20260619 18:51:11.588629 31955 leader_election.cc:302] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77d1cb5e84ca4d62a8b4dc58c64b9a7d, ac0b56b006514a4495e128b32f0493c9; no voters:
I20260619 18:51:11.588819 32353 raft_consensus.cc:2781] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:11.589066 32353 raft_consensus.cc:686] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [term 1 LEADER]: Becoming Leader. State: Replica: 77d1cb5e84ca4d62a8b4dc58c64b9a7d, State: Running, Role: LEADER
I20260619 18:51:11.589411 32353 consensus_queue.cc:237] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } }
I20260619 18:51:11.592815 31875 catalog_manager.cc:5495] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 77d1cb5e84ca4d62a8b4dc58c64b9a7d (127.12.234.193). New cstate: current_term: 1 leader_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77d1cb5e84ca4d62a8b4dc58c64b9a7d" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 44857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 } health_report { overall_health: UNKNOWN } } }
I20260619 18:51:11.609120 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.605468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.12.234.193:44857)
0619 18:51:11.605623 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:11.605628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:11.605646 (+ 18us) server_negotiation.cc:407] Connection header received
0619 18:51:11.605727 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:11.605729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:11.605785 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:11.605876 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:11.606203 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.606723 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.607581 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:11.607798 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:11.607887 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:11.608224 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:11.608680 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:11.608855 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:11.608932 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 18:51:11.608973 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 18:51:11.615350 32151 raft_consensus.cc:1261] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [term 1 FOLLOWER]: Refusing update from remote peer 77d1cb5e84ca4d62a8b4dc58c64b9a7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:11.615350 32282 raft_consensus.cc:1261] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [term 1 FOLLOWER]: Refusing update from remote peer 77d1cb5e84ca4d62a8b4dc58c64b9a7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:11.616134 32353 consensus_queue.cc:1035] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac0b56b006514a4495e128b32f0493c9" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 35387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:11.616369 32356 consensus_queue.cc:1035] T ce05bf0e86954bea9810900c42780f41 P 77d1cb5e84ca4d62a8b4dc58c64b9a7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c61da5761349078acc61eb62bf1f01" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 39547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:11.623595 32359 mvcc.cc:205] Tried to move back new op lower bound from 7298642213328203776 to 7298642213233029120. Current Snapshot: MvccSnapshot[applied={T|T < 7298642213328203776}]
I20260619 18:51:11.626777 32361 mvcc.cc:205] Tried to move back new op lower bound from 7298642213328203776 to 7298642213233029120. Current Snapshot: MvccSnapshot[applied={T|T < 7298642213328203776}]
I20260619 18:51:11.634663 31875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:11.634881 31875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:11.637665 31875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 539ca873880d476fa3788ab5339978bc: Sending DeleteTablet for 3 replicas of tablet ce05bf0e86954bea9810900c42780f41
I20260619 18:51:11.638626 32131 tablet_service.cc:1514] Processing DeleteTablet for tablet ce05bf0e86954bea9810900c42780f41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:11 UTC) from {username='slave'} at 127.0.0.1:46518
I20260619 18:51:11.638778 32000 tablet_service.cc:1514] Processing DeleteTablet for tablet ce05bf0e86954bea9810900c42780f41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:11 UTC) from {username='slave'} at 127.0.0.1:43300
I20260619 18:51:11.639271 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 31938
I20260619 18:51:11.639294 32262 tablet_service.cc:1514] Processing DeleteTablet for tablet ce05bf0e86954bea9810900c42780f41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:11 UTC) from {username='slave'} at 127.0.0.1:39082
I20260619 18:51:11.639551 32369 tablet_replica.cc:330] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: stopping tablet replica
I20260619 18:51:11.639918 32369 raft_consensus.cc:2229] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:51:11.640322 32369 raft_consensus.cc:2258] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:11.640458 32371 tablet_replica.cc:330] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: stopping tablet replica
I20260619 18:51:11.640767 32371 raft_consensus.cc:2229] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:51:11.641044 32371 raft_consensus.cc:2258] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:11.642675 32369 ts_tablet_manager.cc:1905] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:11.642769 32371 ts_tablet_manager.cc:1905] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:11.644961 32371 ts_tablet_manager.cc:1918] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:11.645113 32371 log.cc:1196] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/ce05bf0e86954bea9810900c42780f41
I20260619 18:51:11.645404 32371 ts_tablet_manager.cc:1939] T ce05bf0e86954bea9810900c42780f41 P 04c61da5761349078acc61eb62bf1f01: Deleting consensus metadata
I20260619 18:51:11.645399 32369 ts_tablet_manager.cc:1918] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:11.645486 32369 log.cc:1196] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/ce05bf0e86954bea9810900c42780f41
I20260619 18:51:11.645778 32369 ts_tablet_manager.cc:1939] T ce05bf0e86954bea9810900c42780f41 P ac0b56b006514a4495e128b32f0493c9: Deleting consensus metadata
I20260619 18:51:11.646677 31864 catalog_manager.cc:4841] TS 04c61da5761349078acc61eb62bf1f01 (127.12.234.195:39547): tablet ce05bf0e86954bea9810900c42780f41 (table test-table [id=5eb5b2ff37d64412a04e026bfb34d1e4]) successfully deleted
W20260619 18:51:11.646754 31862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:44857 (error 108)
I20260619 18:51:11.646876 31864 catalog_manager.cc:4841] TS ac0b56b006514a4495e128b32f0493c9 (127.12.234.194:35387): tablet ce05bf0e86954bea9810900c42780f41 (table test-table [id=5eb5b2ff37d64412a04e026bfb34d1e4]) successfully deleted
I20260619 18:51:11.647360 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 32069
I20260619 18:51:11.648295 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:11.647454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:44857 (local address 127.0.0.1:43306)
0619 18:51:11.648157 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:11.648221 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:44857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":617}
W20260619 18:51:11.648455 31862 catalog_manager.cc:4584] TS 77d1cb5e84ca4d62a8b4dc58c64b9a7d (127.12.234.193:44857): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce05bf0e86954bea9810900c42780f41: Network error: Client connection negotiation failed: client connection to 127.12.234.193:44857: connect: Connection refused (error 111)
I20260619 18:51:11.652413 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 32200
I20260619 18:51:11.657254 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 31847
2026-06-19T18:51:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1300 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:11 dist-test-slave-fljv krb5kdc[32375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:11 dist-test-slave-fljv krb5kdc[32375](info): set up 2 sockets
Jun 19 18:51:11 dist-test-slave-fljv krb5kdc[32375](info): commencing operation
krb5kdc: starting...
W20260619 18:51:12.625962 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:12 dist-test-slave-fljv krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:12.749146 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:45279
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:45279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:12.860846 32388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:12.861178 32388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:12.861238 32388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:12.865490 32388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:12.865582 32388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:12.865610 32388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:12.865630 32388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:12.865651 32388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:12.871340 32388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:45279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:45279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:12.872515 32388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:12.873667 32388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:12.882897 32394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:12.883009 32393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:12.883241 32388 server_base.cc:1034] running on GCE node
W20260619 18:51:12.882917 32396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:12.884452 32388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:12.886036 32388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:12.887229 32388 hybrid_clock.cc:648] HybridClock initialized: now 1781895072887209 us; error 42 us; skew 500 ppm
Jun 19 18:51:12 dist-test-slave-fljv krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895072, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:12.890168 32388 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:12.891180 32388 webserver.cc:460] Webserver started at http://127.12.234.254:36561/ using document root <none> and password file <none>
I20260619 18:51:12.891752 32388 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:12.891811 32388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:12.891994 32388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:12.894078 32388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "55fcaefac759478b8495966c3f7e3106"
format_stamp: "Formatted at 2026-06-19 18:51:12 on dist-test-slave-fljv"
server_key: "1a504caaef9dbdca5946c3ebf94fb726"
server_key_iv: "b0a34b2d7e52142dcce52ef78518fa3d"
server_key_version: "encryptionkey@0"
I20260619 18:51:12.894667 32388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "55fcaefac759478b8495966c3f7e3106"
format_stamp: "Formatted at 2026-06-19 18:51:12 on dist-test-slave-fljv"
server_key: "1a504caaef9dbdca5946c3ebf94fb726"
server_key_iv: "b0a34b2d7e52142dcce52ef78518fa3d"
server_key_version: "encryptionkey@0"
I20260619 18:51:12.898707 32388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 18:51:12.901353 32403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:12.902913 32388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:12.903053 32388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "55fcaefac759478b8495966c3f7e3106"
format_stamp: "Formatted at 2026-06-19 18:51:12 on dist-test-slave-fljv"
server_key: "1a504caaef9dbdca5946c3ebf94fb726"
server_key_iv: "b0a34b2d7e52142dcce52ef78518fa3d"
server_key_version: "encryptionkey@0"
I20260619 18:51:12.903182 32388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:12.921159 32388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:12.924983 32388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:12.925194 32388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:12.934501 32388 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:45279
I20260619 18:51:12.934545 32455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:45279 every 8 connection(s)
I20260619 18:51:12.935793 32388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:12.939476 32456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:12.945936 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 32388
I20260619 18:51:12.946095 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:12.946419 13227 external_mini_cluster.cc:1428] Setting key 307a6680c5b797e0736ce9c1d3659d0c
I20260619 18:51:12.949998 32456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106: Bootstrap starting.
I20260619 18:51:12.953053 32456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:12.954054 32456 log.cc:824] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:12.956857 32456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106: No bootstrap required, opened a new log
Jun 19 18:51:12 dist-test-slave-fljv krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:12.960626 32456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fcaefac759478b8495966c3f7e3106" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45279 } }
I20260619 18:51:12.960882 32456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:12.960940 32456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55fcaefac759478b8495966c3f7e3106, State: Initialized, Role: FOLLOWER
I20260619 18:51:12.961455 32456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fcaefac759478b8495966c3f7e3106" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45279 } }
I20260619 18:51:12.961621 32456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:12.961689 32456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:12.961791 32456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:12.962903 32456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fcaefac759478b8495966c3f7e3106" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45279 } }
I20260619 18:51:12.963299 32456 leader_election.cc:302] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55fcaefac759478b8495966c3f7e3106; no voters:
I20260619 18:51:12.963640 32456 leader_election.cc:288] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:12.963732 32461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:12.963943 32461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [term 1 LEADER]: Becoming Leader. State: Replica: 55fcaefac759478b8495966c3f7e3106, State: Running, Role: LEADER
I20260619 18:51:12.964234 32459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:12.947998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52716 (local address 127.12.234.254:45279)
0619 18:51:12.948509 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:12.948532 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:12.948587 (+ 55us) server_negotiation.cc:407] Connection header received
0619 18:51:12.949566 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:12.949596 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:12.950130 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:12.951501 (+ 1371us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:12.951987 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:12.953582 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:12.954886 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:12.955599 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:12.955808 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:12.958752 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:12.958758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:12.958776 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:12.958804 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:12.961028 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:12.961572 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:12.961577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:12.961582 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:12.961730 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:12.962015 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:12.962018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:12.962019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:12.962477 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:12.962643 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:12.962937 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 18:51:12.963247 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":122,"threads_started":1}
I20260619 18:51:12.964294 32461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fcaefac759478b8495966c3f7e3106" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45279 } }
I20260619 18:51:12.964927 32456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:12.965505 32461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55fcaefac759478b8495966c3f7e3106. Latest consensus state: current_term: 1 leader_uuid: "55fcaefac759478b8495966c3f7e3106" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fcaefac759478b8495966c3f7e3106" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45279 } } }
I20260619 18:51:12.965629 32461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:12.967397 32462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55fcaefac759478b8495966c3f7e3106" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fcaefac759478b8495966c3f7e3106" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 45279 } } }
I20260619 18:51:12.967173 32467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:12.967573 32462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:12.970155 32467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:12.975561 32467 catalog_manager.cc:1347] Generated new cluster ID: cddfabf8577a45a1809943d6d00bf01c
I20260619 18:51:12.975641 32467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:12.987540 32467 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:12.988940 32467 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:12.999678 32467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106: Generated new TSK 0
I20260619 18:51:13.000669 32467 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
Entry for principal kudu/127.12.234.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab.
I20260619 18:51:13.037707 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45279
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:13.153476 32482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:13.153780 32482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:13.153841 32482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:13.157963 32482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:13.158044 32482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:13.158133 32482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:51:13.163739 32482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.193.keytab
--principal=kudu/127.12.234.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:13.164889 32482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:13.166074 32482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:13.176308 32487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:13.176318 32488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:13.176318 32490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:13.176772 32482 server_base.cc:1034] running on GCE node
I20260619 18:51:13.177268 32482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:13.178045 32482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:13.179201 32482 hybrid_clock.cc:648] HybridClock initialized: now 1781895073179169 us; error 49 us; skew 500 ppm
Jun 19 18:51:13 dist-test-slave-fljv krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895073, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:13.182085 32482 init.cc:377] Logged in from keytab as kudu/127.12.234.193@KRBTEST.COM (short username kudu)
I20260619 18:51:13.183075 32482 webserver.cc:460] Webserver started at http://127.12.234.193:43189/ using document root <none> and password file <none>
I20260619 18:51:13.183648 32482 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:13.183712 32482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:13.183938 32482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:13.186093 32482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "81d8e4a3e1b64419a81a1d39f2415882"
format_stamp: "Formatted at 2026-06-19 18:51:13 on dist-test-slave-fljv"
server_key: "69a6b000b0507cb795df9d7cf1738725"
server_key_iv: "0fc98d754ae4217dac71ec7bb7fc296e"
server_key_version: "encryptionkey@0"
I20260619 18:51:13.186717 32482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "81d8e4a3e1b64419a81a1d39f2415882"
format_stamp: "Formatted at 2026-06-19 18:51:13 on dist-test-slave-fljv"
server_key: "69a6b000b0507cb795df9d7cf1738725"
server_key_iv: "0fc98d754ae4217dac71ec7bb7fc296e"
server_key_version: "encryptionkey@0"
I20260619 18:51:13.190689 32482 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:13.193234 32497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:13.194661 32482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:13.194779 32482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "81d8e4a3e1b64419a81a1d39f2415882"
format_stamp: "Formatted at 2026-06-19 18:51:13 on dist-test-slave-fljv"
server_key: "69a6b000b0507cb795df9d7cf1738725"
server_key_iv: "0fc98d754ae4217dac71ec7bb7fc296e"
server_key_version: "encryptionkey@0"
I20260619 18:51:13.194886 32482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:13.206002 32482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:13.209319 32482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:13.209501 32482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:13.210098 32482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:13.210968 32482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:13.211014 32482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:13.211067 32482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:13.211100 32482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:13.219491 32482 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:41035
I20260619 18:51:13.219563 32610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:41035 every 8 connection(s)
I20260619 18:51:13.220454 32482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:51:13.224893 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 32482
I20260619 18:51:13.225037 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:51:13.225351 13227 external_mini_cluster.cc:1428] Setting key 438c9a2a9a7a569dbff5b756db59ad0f
Jun 19 18:51:13 dist-test-slave-fljv krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895073, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.193@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.194@KRBTEST.COM; defaulting to no policy
I20260619 18:51:13.235889 32459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.222428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:59545 (local address 127.12.234.254:45279)
0619 18:51:13.222618 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.222623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.223560 (+ 937us) server_negotiation.cc:407] Connection header received
0619 18:51:13.225066 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.225070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.225131 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.225211 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:13.226631 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.227331 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.228558 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.228924 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.229043 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.231832 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:13.231836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:13.231838 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:13.231863 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:13.233388 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:13.234083 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:13.234087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:13.234088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:13.234206 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:13.234669 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:13.234672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:13.234674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:13.234888 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:13.234968 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.235590 (+ 622us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.235746 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 18:51:13.236793 32613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.222887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:45279 (local address 127.12.234.193:59545)
0619 18:51:13.223382 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:13.223430 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:13.224777 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:13.225450 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:13.225460 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:13.225917 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:13.226477 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.226491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.227474 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.227477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.228425 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.228437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.229123 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.229126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.229289 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.229875 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:13.229898 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:13.231629 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:13.233526 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:13.233533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:13.233550 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:13.233940 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:13.234295 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:13.234299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:13.234301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:13.234517 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:13.235012 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:13.235018 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:13.235293 (+ 275us) client_negotiation.cc:769] Sending connection context
0619 18:51:13.235506 (+ 213us) client_negotiation.cc:240] Negotiation successful
0619 18:51:13.235834 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":134,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.194@KRBTEST.COM" created.
I20260619 18:51:13.238384 32611 heartbeater.cc:347] Connected to a master server at 127.12.234.254:45279
I20260619 18:51:13.238641 32611 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:13.239158 32611 heartbeater.cc:511] Master 127.12.234.254:45279 requested a full tablet report, sending...
I20260619 18:51:13.240783 32420 ts_manager.cc:194] Registered new tserver with Master: 81d8e4a3e1b64419a81a1d39f2415882 (127.12.234.193:41035)
I20260619 18:51:13.241973 32420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.193@KRBTEST.COM'} at 127.12.234.193:59545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
Entry for principal kudu/127.12.234.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab.
I20260619 18:51:13.252728 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45279
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:13.367270 32616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:13.367573 32616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:13.367635 32616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:13.371809 32616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:13.371886 32616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:13.371973 32616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:51:13.377525 32616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.194.keytab
--principal=kudu/127.12.234.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:13.378717 32616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:13.379786 32616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:13.389411 32622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:13.389438 32624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:13.389417 32621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:13.389814 32616 server_base.cc:1034] running on GCE node
I20260619 18:51:13.390314 32616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:13.390982 32616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:13.392170 32616 hybrid_clock.cc:648] HybridClock initialized: now 1781895073392151 us; error 36 us; skew 500 ppm
Jun 19 18:51:13 dist-test-slave-fljv krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895073, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:13.395290 32616 init.cc:377] Logged in from keytab as kudu/127.12.234.194@KRBTEST.COM (short username kudu)
I20260619 18:51:13.396322 32616 webserver.cc:460] Webserver started at http://127.12.234.194:41133/ using document root <none> and password file <none>
I20260619 18:51:13.396895 32616 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:13.396972 32616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:13.397197 32616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:13.399561 32616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "d8ccd472291c4e37a8264181793083b1"
format_stamp: "Formatted at 2026-06-19 18:51:13 on dist-test-slave-fljv"
server_key: "8f6893d61cfa85f7e00abfaad63ca49e"
server_key_iv: "132052a5d271ec2a5a3227983bc9950c"
server_key_version: "encryptionkey@0"
I20260619 18:51:13.400202 32616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "d8ccd472291c4e37a8264181793083b1"
format_stamp: "Formatted at 2026-06-19 18:51:13 on dist-test-slave-fljv"
server_key: "8f6893d61cfa85f7e00abfaad63ca49e"
server_key_iv: "132052a5d271ec2a5a3227983bc9950c"
server_key_version: "encryptionkey@0"
I20260619 18:51:13.404379 32616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260619 18:51:13.407021 32631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:13.408478 32616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:13.408588 32616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "d8ccd472291c4e37a8264181793083b1"
format_stamp: "Formatted at 2026-06-19 18:51:13 on dist-test-slave-fljv"
server_key: "8f6893d61cfa85f7e00abfaad63ca49e"
server_key_iv: "132052a5d271ec2a5a3227983bc9950c"
server_key_version: "encryptionkey@0"
I20260619 18:51:13.408723 32616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:13.421255 32616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:13.424494 32616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:13.424656 32616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:13.425211 32616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:13.426009 32616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:13.426055 32616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:13.426095 32616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:13.426107 32616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:13.434633 32616 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:36627
I20260619 18:51:13.434659 32744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:36627 every 8 connection(s)
I20260619 18:51:13.435549 32616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:51:13.439355 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 32616
I20260619 18:51:13.439482 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:51:13.439805 13227 external_mini_cluster.cc:1428] Setting key a542b9fc36d0afddca209580fc168eb4
Jun 19 18:51:13 dist-test-slave-fljv krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895073, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.194@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.195@KRBTEST.COM; defaulting to no policy
I20260619 18:51:13.449838 32459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.437332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:41571 (local address 127.12.234.254:45279)
0619 18:51:13.437469 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.437473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.438391 (+ 918us) server_negotiation.cc:407] Connection header received
0619 18:51:13.439947 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.439950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.440000 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.440076 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:13.441435 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.441964 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.443434 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.443689 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.443781 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.446282 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:13.446286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:13.446288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:13.446314 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:13.447626 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:13.448147 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:13.448150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:13.448151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:13.448235 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:13.448614 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:13.448617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:13.448619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:13.448829 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:13.448915 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.449535 (+ 620us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.449673 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.195@KRBTEST.COM" created.
I20260619 18:51:13.450906 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.437721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:45279 (local address 127.12.234.194:41571)
0619 18:51:13.438238 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:13.438282 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:13.439711 (+ 1429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:13.440263 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:13.440270 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:13.440703 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:13.441281 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.441302 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.442094 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.442098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.443310 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.443321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.443839 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.443843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.444012 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.444562 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:13.444579 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:13.446072 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:13.447756 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:13.447763 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:13.447781 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:13.448040 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:13.448319 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:13.448322 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:13.448324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:13.448510 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:13.448930 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:13.448937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:13.449285 (+ 348us) client_negotiation.cc:769] Sending connection context
0619 18:51:13.449473 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 18:51:13.449829 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":119,"threads_started":1}
I20260619 18:51:13.452175 32745 heartbeater.cc:347] Connected to a master server at 127.12.234.254:45279
I20260619 18:51:13.452417 32745 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:13.452939 32745 heartbeater.cc:511] Master 127.12.234.254:45279 requested a full tablet report, sending...
I20260619 18:51:13.454051 32420 ts_manager.cc:194] Registered new tserver with Master: d8ccd472291c4e37a8264181793083b1 (127.12.234.194:36627)
I20260619 18:51:13.454873 32420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.194@KRBTEST.COM'} at 127.12.234.194:41571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
Entry for principal kudu/127.12.234.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab.
I20260619 18:51:13.466266 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45279
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:13.578102 32750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:13.578398 32750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:13.578446 32750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:13.582579 32750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:13.582644 32750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:13.582712 32750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:51:13.588455 32750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.195.keytab
--principal=kudu/127.12.234.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:45279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:13.589589 32750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:13.590909 32750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:13.602756 32758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:13.602748 32756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:13.602753 32755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:13.602823 32750 server_base.cc:1034] running on GCE node
I20260619 18:51:13.603387 32750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:13.604020 32750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:13.605212 32750 hybrid_clock.cc:648] HybridClock initialized: now 1781895073605194 us; error 43 us; skew 500 ppm
Jun 19 18:51:13 dist-test-slave-fljv krb5kdc[32375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895073, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:13.608155 32750 init.cc:377] Logged in from keytab as kudu/127.12.234.195@KRBTEST.COM (short username kudu)
I20260619 18:51:13.609083 32750 webserver.cc:460] Webserver started at http://127.12.234.195:41789/ using document root <none> and password file <none>
I20260619 18:51:13.609623 32750 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:13.609685 32750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:13.609874 32750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:13.612023 32750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "d7f4e129f6e84ee78c96e939daba54a5"
format_stamp: "Formatted at 2026-06-19 18:51:13 on dist-test-slave-fljv"
server_key: "5ad47678de51a1cbf1e7bcf2a98346ae"
server_key_iv: "b7a4ccf5a5e6d16221c94ef4c8bd0aae"
server_key_version: "encryptionkey@0"
I20260619 18:51:13.612592 32750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "d7f4e129f6e84ee78c96e939daba54a5"
format_stamp: "Formatted at 2026-06-19 18:51:13 on dist-test-slave-fljv"
server_key: "5ad47678de51a1cbf1e7bcf2a98346ae"
server_key_iv: "b7a4ccf5a5e6d16221c94ef4c8bd0aae"
server_key_version: "encryptionkey@0"
I20260619 18:51:13.616474 32750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:51:13.618942 32765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:13.620422 32750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:13.620543 32750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "d7f4e129f6e84ee78c96e939daba54a5"
format_stamp: "Formatted at 2026-06-19 18:51:13 on dist-test-slave-fljv"
server_key: "5ad47678de51a1cbf1e7bcf2a98346ae"
server_key_iv: "b7a4ccf5a5e6d16221c94ef4c8bd0aae"
server_key_version: "encryptionkey@0"
I20260619 18:51:13.620661 32750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:13.644701 32750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:13.648000 32750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:13.648190 32750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:13.648808 32750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:13.649675 32750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:13.649722 32750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:13.649768 32750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:13.649796 32750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:13.658489 32750 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:45235
I20260619 18:51:13.658524 410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:45235 every 8 connection(s)
I20260619 18:51:13.659402 32750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:51:13.663278 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 32750
I20260619 18:51:13.663395 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:51:13.663676 13227 external_mini_cluster.cc:1428] Setting key 70fe5c52f47b8be1dbcd96d883a96c84
Jun 19 18:51:13 dist-test-slave-fljv krb5kdc[32375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895073, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.195@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:13.674288 32459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.661253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:51819 (local address 127.12.234.254:45279)
0619 18:51:13.661422 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.661427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.662343 (+ 916us) server_negotiation.cc:407] Connection header received
0619 18:51:13.663792 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.663796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.663851 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.663946 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:13.665369 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.666138 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.667435 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.667664 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.667745 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.670424 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:13.670428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:13.670430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:13.670460 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:13.671860 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:13.672425 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:13.672429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:13.672430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:13.672564 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:13.673025 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:13.673028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:13.673029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:13.673254 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:13.673347 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.673932 (+ 585us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.674103 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 18:51:13.675143 413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.661662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:45279 (local address 127.12.234.195:51819)
0619 18:51:13.662141 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:13.662232 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:13.663562 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:13.664146 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:13.664157 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:13.664594 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:13.665242 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.665259 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.666274 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.666277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.667321 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.667331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.667814 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.667817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.667992 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.668535 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:13.668551 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:13.670211 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:13.671992 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:13.671998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:13.672014 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:13.672303 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:13.672688 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:13.672691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:13.672693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:13.672874 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:13.673362 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:13.673367 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:13.673665 (+ 298us) client_negotiation.cc:769] Sending connection context
0619 18:51:13.673842 (+ 177us) client_negotiation.cc:240] Negotiation successful
0619 18:51:13.674164 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":121,"threads_started":1}
I20260619 18:51:13.676517 411 heartbeater.cc:347] Connected to a master server at 127.12.234.254:45279
I20260619 18:51:13.676801 411 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:13.677413 411 heartbeater.cc:511] Master 127.12.234.254:45279 requested a full tablet report, sending...
I20260619 18:51:13.678508 32420 ts_manager.cc:194] Registered new tserver with Master: d7f4e129f6e84ee78c96e939daba54a5 (127.12.234.195:45235)
I20260619 18:51:13.679071 32420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.195@KRBTEST.COM'} at 127.12.234.195:51819
I20260619 18:51:13.690106 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:51:13.699471 32459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.692145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52720 (local address 127.12.234.254:45279)
0619 18:51:13.692267 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.692272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.692468 (+ 196us) server_negotiation.cc:407] Connection header received
0619 18:51:13.692595 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.692598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.692651 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.692731 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:13.693438 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.693954 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.694850 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.695066 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.695196 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.696500 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:13.696503 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:13.696505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:13.696533 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:13.697830 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:13.698252 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:13.698256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:13.698258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:13.698339 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:13.698596 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:13.698599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:13.698601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:13.698832 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:13.698949 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.699199 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.699319 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 18:51:13.702445 32420 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:51:13.705423 32420 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:51:13.720356 424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.714433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.12.234.193:41035)
0619 18:51:13.714932 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.714949 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.715644 (+ 695us) server_negotiation.cc:407] Connection header received
0619 18:51:13.715847 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.715853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.716031 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.716150 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:13.716522 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.717248 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.718843 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.719658 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.719785 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.719937 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.720090 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.720206 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":99,"threads_started":1}
I20260619 18:51:13.720350 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.714215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:36627 (local address 127.0.0.1:45170)
0619 18:51:13.715031 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:13.715086 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:13.715207 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:13.715811 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:13.715817 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:13.715839 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:13.715984 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.715991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.717474 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.717477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.718891 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.718901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.720012 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.720015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.720119 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.720140 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 18:51:13.720198 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 18:51:13.720251 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":64,"thread_start_us":83,"threads_started":1}
I20260619 18:51:13.720809 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.715035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.12.234.195:45235)
0619 18:51:13.715410 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.715420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.715681 (+ 261us) server_negotiation.cc:407] Connection header received
0619 18:51:13.715750 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.715756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.715944 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.716102 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:13.716318 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.717393 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.719178 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.720006 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.720119 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.720240 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.720622 (+ 382us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.720707 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":122,"threads_started":1}
I20260619 18:51:13.720348 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.714738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:41035 (local address 127.0.0.1:57112)
0619 18:51:13.715537 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:13.715549 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:13.715712 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:13.716342 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:13.716344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:13.716360 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:13.716422 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.716426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.717374 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.717378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.718727 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.718736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.719784 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.719788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.719893 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.719930 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 18:51:13.720045 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 18:51:13.720181 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":123,"thread_start_us":93,"threads_started":1}
I20260619 18:51:13.720937 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.714323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.12.234.194:36627)
0619 18:51:13.715125 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.715137 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.715163 (+ 26us) server_negotiation.cc:407] Connection header received
0619 18:51:13.715423 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.715429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.715657 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.715819 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:13.716120 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.717343 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.719036 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.719674 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.719785 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.719937 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.720684 (+ 747us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.720789 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":117,"threads_started":1}
I20260619 18:51:13.721800 426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.714498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:45235 (local address 127.0.0.1:44268)
0619 18:51:13.715474 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:13.715484 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:13.715589 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:13.716083 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:13.716086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:13.716102 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:13.716212 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.716218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.717524 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.717528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.719058 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.719067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.720304 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.720307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.720392 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.720414 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:51:13.720479 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 18:51:13.720546 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":206,"thread_start_us":95,"threads_started":1}
I20260619 18:51:13.723811 32679 tablet_service.cc:1467] Processing CreateTablet for tablet d13817db7755424b96f6062f3cace8af (DEFAULT_TABLE table=test-table [id=944d868e7b0b459e9492dc5527aa783d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:13.724318 32545 tablet_service.cc:1467] Processing CreateTablet for tablet d13817db7755424b96f6062f3cace8af (DEFAULT_TABLE table=test-table [id=944d868e7b0b459e9492dc5527aa783d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:13.724350 345 tablet_service.cc:1467] Processing CreateTablet for tablet d13817db7755424b96f6062f3cace8af (DEFAULT_TABLE table=test-table [id=944d868e7b0b459e9492dc5527aa783d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:13.724800 32679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d13817db7755424b96f6062f3cace8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:13.725389 32545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d13817db7755424b96f6062f3cace8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:13.725389 345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d13817db7755424b96f6062f3cace8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:13.731671 428 tablet_bootstrap.cc:492] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: Bootstrap starting.
I20260619 18:51:13.732576 429 tablet_bootstrap.cc:492] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: Bootstrap starting.
I20260619 18:51:13.733335 430 tablet_bootstrap.cc:492] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882: Bootstrap starting.
I20260619 18:51:13.734700 428 tablet_bootstrap.cc:654] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:13.734700 429 tablet_bootstrap.cc:654] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:13.735168 430 tablet_bootstrap.cc:654] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:13.736063 428 log.cc:824] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:13.736071 429 log.cc:824] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:13.736382 430 log.cc:824] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:13.738325 430 tablet_bootstrap.cc:492] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882: No bootstrap required, opened a new log
I20260619 18:51:13.738325 429 tablet_bootstrap.cc:492] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: No bootstrap required, opened a new log
I20260619 18:51:13.738535 430 ts_tablet_manager.cc:1397] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260619 18:51:13.738535 429 ts_tablet_manager.cc:1397] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:51:13.739354 428 tablet_bootstrap.cc:492] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: No bootstrap required, opened a new log
I20260619 18:51:13.739539 428 ts_tablet_manager.cc:1397] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260619 18:51:13.742408 429 raft_consensus.cc:348] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } }
I20260619 18:51:13.742412 428 raft_consensus.cc:348] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } }
I20260619 18:51:13.742660 429 raft_consensus.cc:374] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:13.742707 428 raft_consensus.cc:374] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:13.742713 429 raft_consensus.cc:729] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f4e129f6e84ee78c96e939daba54a5, State: Initialized, Role: FOLLOWER
I20260619 18:51:13.742758 428 raft_consensus.cc:729] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8ccd472291c4e37a8264181793083b1, State: Initialized, Role: FOLLOWER
I20260619 18:51:13.743176 429 consensus_queue.cc:260] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } }
I20260619 18:51:13.743237 428 consensus_queue.cc:260] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } }
I20260619 18:51:13.743518 430 raft_consensus.cc:348] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } }
I20260619 18:51:13.743767 430 raft_consensus.cc:374] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:13.743811 430 raft_consensus.cc:729] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d8e4a3e1b64419a81a1d39f2415882, State: Initialized, Role: FOLLOWER
I20260619 18:51:13.743839 411 heartbeater.cc:503] Master 127.12.234.254:45279 was elected leader, sending a full tablet report...
I20260619 18:51:13.744212 429 ts_tablet_manager.cc:1428] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 18:51:13.744215 428 ts_tablet_manager.cc:1428] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 18:51:13.744244 430 consensus_queue.cc:260] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } }
I20260619 18:51:13.744540 32745 heartbeater.cc:503] Master 127.12.234.254:45279 was elected leader, sending a full tablet report...
I20260619 18:51:13.744814 32611 heartbeater.cc:503] Master 127.12.234.254:45279 was elected leader, sending a full tablet report...
I20260619 18:51:13.745087 430 ts_tablet_manager.cc:1428] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260619 18:51:13.910414 412 tablet.cc:2369] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:13.922976 435 raft_consensus.cc:482] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:51:13.923219 435 raft_consensus.cc:504] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } }
I20260619 18:51:13.925108 435 leader_election.cc:288] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81d8e4a3e1b64419a81a1d39f2415882 (127.12.234.193:41035), d7f4e129f6e84ee78c96e939daba54a5 (127.12.234.195:45235)
I20260619 18:51:13.929467 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.924874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:41035 (local address 127.12.234.194:53701)
0619 18:51:13.925242 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:13.925259 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:13.925382 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:13.925750 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:13.925753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:13.925773 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:13.925950 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.925957 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.926999 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.927002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.928217 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.928228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.929041 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.929045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.929155 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.929180 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 18:51:13.929254 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 18:51:13.929327 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271}
I20260619 18:51:13.929962 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.925516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:45235 (local address 127.12.234.194:59919)
0619 18:51:13.925879 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:13.925891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:13.925981 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:13.926277 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:13.926306 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:13.926324 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:13.926464 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.926471 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.927503 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.927506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.928678 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:13.928688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.929550 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.929553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:13.929647 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.929669 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 18:51:13.929767 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 18:51:13.929830 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":86,"threads_started":1}
I20260619 18:51:13.929975 424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.925056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:53701 (local address 127.12.234.193:41035)
0619 18:51:13.925248 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.925253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.925322 (+ 69us) server_negotiation.cc:407] Connection header received
0619 18:51:13.925510 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.925513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.925600 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.926058 (+ 458us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:13.926163 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.926891 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.928342 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.928937 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.929025 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.929078 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.929751 (+ 673us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.929830 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 18:51:13.930016 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.925641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:59919 (local address 127.12.234.195:45235)
0619 18:51:13.925763 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.925767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.925954 (+ 187us) server_negotiation.cc:407] Connection header received
0619 18:51:13.926085 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.926088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.926140 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.926242 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:13.926589 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.927366 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.928806 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.929347 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.929445 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.929491 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.929800 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.929889 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:51:13.930738 365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d13817db7755424b96f6062f3cace8af" candidate_uuid: "d8ccd472291c4e37a8264181793083b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f4e129f6e84ee78c96e939daba54a5" is_pre_election: true
I20260619 18:51:13.930794 32565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d13817db7755424b96f6062f3cace8af" candidate_uuid: "d8ccd472291c4e37a8264181793083b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81d8e4a3e1b64419a81a1d39f2415882" is_pre_election: true
I20260619 18:51:13.931077 365 raft_consensus.cc:2454] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8ccd472291c4e37a8264181793083b1 in term 0.
I20260619 18:51:13.931082 32565 raft_consensus.cc:2454] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8ccd472291c4e37a8264181793083b1 in term 0.
I20260619 18:51:13.931578 32633 leader_election.cc:302] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7f4e129f6e84ee78c96e939daba54a5, d8ccd472291c4e37a8264181793083b1; no voters:
I20260619 18:51:13.932035 435 raft_consensus.cc:2781] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:51:13.932143 435 raft_consensus.cc:482] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:51:13.932194 435 raft_consensus.cc:3037] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:13.933531 435 raft_consensus.cc:504] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } }
I20260619 18:51:13.933909 435 leader_election.cc:288] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [CANDIDATE]: Term 1 election: Requested vote from peers 81d8e4a3e1b64419a81a1d39f2415882 (127.12.234.193:41035), d7f4e129f6e84ee78c96e939daba54a5 (127.12.234.195:45235)
I20260619 18:51:13.934316 32565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d13817db7755424b96f6062f3cace8af" candidate_uuid: "d8ccd472291c4e37a8264181793083b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81d8e4a3e1b64419a81a1d39f2415882"
I20260619 18:51:13.934315 365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d13817db7755424b96f6062f3cace8af" candidate_uuid: "d8ccd472291c4e37a8264181793083b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f4e129f6e84ee78c96e939daba54a5"
I20260619 18:51:13.934448 32565 raft_consensus.cc:3037] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:13.934449 365 raft_consensus.cc:3037] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:13.935765 365 raft_consensus.cc:2454] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8ccd472291c4e37a8264181793083b1 in term 1.
I20260619 18:51:13.935765 32565 raft_consensus.cc:2454] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8ccd472291c4e37a8264181793083b1 in term 1.
I20260619 18:51:13.936086 32633 leader_election.cc:302] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7f4e129f6e84ee78c96e939daba54a5, d8ccd472291c4e37a8264181793083b1; no voters:
I20260619 18:51:13.936304 435 raft_consensus.cc:2781] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:13.936534 435 raft_consensus.cc:686] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 1 LEADER]: Becoming Leader. State: Replica: d8ccd472291c4e37a8264181793083b1, State: Running, Role: LEADER
W20260619 18:51:13.936610 32746 tablet.cc:2369] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:13.936849 435 consensus_queue.cc:237] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } }
I20260619 18:51:13.940125 32419 catalog_manager.cc:5495] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8ccd472291c4e37a8264181793083b1 (127.12.234.194). New cstate: current_term: 1 leader_uuid: "d8ccd472291c4e37a8264181793083b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8ccd472291c4e37a8264181793083b1" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 36627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 } health_report { overall_health: UNKNOWN } } }
I20260619 18:51:13.961702 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.958214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.12.234.194:36627)
0619 18:51:13.958345 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:13.958349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:13.958364 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:51:13.958487 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:13.958489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:13.958552 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:13.958688 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:13.958922 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.959451 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.960301 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:13.960569 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:13.960659 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:13.960896 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:13.961342 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:13.961463 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:13.961500 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 18:51:13.961555 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 18:51:13.967871 32565 raft_consensus.cc:1261] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882 [term 1 FOLLOWER]: Refusing update from remote peer d8ccd472291c4e37a8264181793083b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:13.967869 365 raft_consensus.cc:1261] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [term 1 FOLLOWER]: Refusing update from remote peer d8ccd472291c4e37a8264181793083b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:13.968576 438 consensus_queue.cc:1035] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81d8e4a3e1b64419a81a1d39f2415882" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 41035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:13.968770 435 consensus_queue.cc:1035] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f4e129f6e84ee78c96e939daba54a5" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 45235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 18:51:13.971764 32612 tablet.cc:2369] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:13.979568 442 mvcc.cc:205] Tried to move back new op lower bound from 7298642222965141504 to 7298642222848823296. Current Snapshot: MvccSnapshot[applied={T|T < 7298642222965141504}]
I20260619 18:51:13.984145 32418 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:13.984367 32418 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:13.987257 32418 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55fcaefac759478b8495966c3f7e3106: Sending DeleteTablet for 3 replicas of tablet d13817db7755424b96f6062f3cace8af
I20260619 18:51:13.988227 32545 tablet_service.cc:1514] Processing DeleteTablet for tablet d13817db7755424b96f6062f3cace8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:13 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:57112
I20260619 18:51:13.988440 32679 tablet_service.cc:1514] Processing DeleteTablet for tablet d13817db7755424b96f6062f3cace8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:13 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:45170
I20260619 18:51:13.988787 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 32482
I20260619 18:51:13.988788 450 tablet_replica.cc:330] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882: stopping tablet replica
I20260619 18:51:13.988997 345 tablet_service.cc:1514] Processing DeleteTablet for tablet d13817db7755424b96f6062f3cace8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:13 UTC) from {username='kudu', principal='kudu/127.12.234.254@KRBTEST.COM'} at 127.0.0.1:44268
I20260619 18:51:13.989137 450 raft_consensus.cc:2229] T d13817db7755424b96f6062f3cace8af P 81d8e4a3e1b64419a81a1d39f2415882 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:51:13.989696 451 tablet_replica.cc:330] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: stopping tablet replica
I20260619 18:51:13.989818 452 tablet_replica.cc:330] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: stopping tablet replica
I20260619 18:51:13.990130 451 raft_consensus.cc:2229] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 18:51:13.990137 452 raft_consensus.cc:2229] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:51:13.990500 452 raft_consensus.cc:2258] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:13.990718 451 raft_consensus.cc:2258] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:13.992400 452 ts_tablet_manager.cc:1905] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:13.992660 451 ts_tablet_manager.cc:1905] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:13.995270 452 ts_tablet_manager.cc:1918] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:13.995383 452 log.cc:1196] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/d13817db7755424b96f6062f3cace8af
I20260619 18:51:13.995702 452 ts_tablet_manager.cc:1939] T d13817db7755424b96f6062f3cace8af P d7f4e129f6e84ee78c96e939daba54a5: Deleting consensus metadata
I20260619 18:51:13.995868 451 ts_tablet_manager.cc:1918] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:13.996038 451 log.cc:1196] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781894826428600-13227-0/minicluster-data/ts-1/wal/wals/d13817db7755424b96f6062f3cace8af
I20260619 18:51:13.996459 451 ts_tablet_manager.cc:1939] T d13817db7755424b96f6062f3cace8af P d8ccd472291c4e37a8264181793083b1: Deleting consensus metadata
I20260619 18:51:13.997161 32405 catalog_manager.cc:4841] TS d7f4e129f6e84ee78c96e939daba54a5 (127.12.234.195:45235): tablet d13817db7755424b96f6062f3cace8af (table test-table [id=944d868e7b0b459e9492dc5527aa783d]) successfully deleted
I20260619 18:51:13.998008 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 32616
W20260619 18:51:13.997972 32407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:41035 (error 108)
I20260619 18:51:13.998085 32406 catalog_manager.cc:4841] TS d8ccd472291c4e37a8264181793083b1 (127.12.234.194:36627): tablet d13817db7755424b96f6062f3cace8af (table test-table [id=944d868e7b0b459e9492dc5527aa783d]) successfully deleted
I20260619 18:51:14.000098 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:13.999769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:41035 (local address 127.0.0.1:57122)
0619 18:51:13.999908 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:13.999995 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:41035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 18:51:14.000313 32407 catalog_manager.cc:4584] TS 81d8e4a3e1b64419a81a1d39f2415882 (127.12.234.193:41035): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d13817db7755424b96f6062f3cace8af: Network error: Client connection negotiation failed: client connection to 127.12.234.193:41035: connect: Connection refused (error 111)
I20260619 18:51:14.003650 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 32750
I20260619 18:51:14.008903 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 32388
2026-06-19T18:51:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2352 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T18:51:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:14Z Disabled control of system clock
I20260619 18:51:14.038489 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40607
--webserver_interface=127.12.234.254
--webserver_port=0
--builtin_ntp_servers=127.12.234.212:39159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:40607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 18:51:14.149099 457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:14.149386 457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:14.149421 457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:14.153512 457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:14.153579 457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:14.153594 457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:14.153609 457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:14.153623 457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:14.158425 457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:39159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:40607
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:40607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:14.159497 457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:14.160859 457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:51:14.169257 457 server_base.cc:1034] running on GCE node
W20260619 18:51:14.169202 463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:14.169202 462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:14.169238 465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:14.170161 457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:14.171343 457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:14.172529 457 hybrid_clock.cc:648] HybridClock initialized: now 1781895074172501 us; error 47 us; skew 500 ppm
I20260619 18:51:14.174597 457 webserver.cc:460] Webserver started at http://127.12.234.254:42139/ using document root <none> and password file <none>
I20260619 18:51:14.175168 457 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:14.175239 457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:14.175470 457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:14.177794 457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "510426bca4134dd683867658c93cf839"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "f963854e78778f795243c960d6a860b7"
server_key_iv: "fe11409e6d4f9aaf075b4d1db7523279"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.178397 457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "510426bca4134dd683867658c93cf839"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "f963854e78778f795243c960d6a860b7"
server_key_iv: "fe11409e6d4f9aaf075b4d1db7523279"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.182353 457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 18:51:14.184945 471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.186311 457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:14.186434 457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "510426bca4134dd683867658c93cf839"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "f963854e78778f795243c960d6a860b7"
server_key_iv: "fe11409e6d4f9aaf075b4d1db7523279"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.186547 457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:14.209353 457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:14.210119 457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:14.210335 457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:14.217351 457 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:40607
I20260619 18:51:14.217375 523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:40607 every 8 connection(s)
I20260619 18:51:14.218357 457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:14.221556 524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:14.224359 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 457
I20260619 18:51:14.224493 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:14.224800 13227 external_mini_cluster.cc:1428] Setting key d349af64525da5537869e34afc824a9d
I20260619 18:51:14.227847 524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839: Bootstrap starting.
I20260619 18:51:14.230425 524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:14.231257 524 log.cc:824] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:14.233340 524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839: No bootstrap required, opened a new log
I20260619 18:51:14.234508 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.226311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39582 (local address 127.12.234.254:40607)
0619 18:51:14.226943 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.226967 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.227007 (+ 40us) server_negotiation.cc:407] Connection header received
0619 18:51:14.227658 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.227676 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.228130 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.228424 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:14.228823 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.229829 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.231089 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.231631 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.231799 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.232546 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:14.232551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:14.232565 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:14.232943 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:14.233042 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:14.233067 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:14.233153 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.233305 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.233691 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
I20260619 18:51:14.236980 524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "510426bca4134dd683867658c93cf839" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40607 } }
I20260619 18:51:14.237198 524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:14.237244 524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 510426bca4134dd683867658c93cf839, State: Initialized, Role: FOLLOWER
I20260619 18:51:14.237654 524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "510426bca4134dd683867658c93cf839" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40607 } }
I20260619 18:51:14.237774 524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:14.237831 524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:14.237915 524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:14.238811 524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "510426bca4134dd683867658c93cf839" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40607 } }
I20260619 18:51:14.239125 524 leader_election.cc:302] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 510426bca4134dd683867658c93cf839; no voters:
I20260619 18:51:14.239410 524 leader_election.cc:288] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:14.239531 529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:14.239692 529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [term 1 LEADER]: Becoming Leader. State: Replica: 510426bca4134dd683867658c93cf839, State: Running, Role: LEADER
I20260619 18:51:14.239980 529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "510426bca4134dd683867658c93cf839" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40607 } }
I20260619 18:51:14.240666 524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:14.241571 531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 510426bca4134dd683867658c93cf839. Latest consensus state: current_term: 1 leader_uuid: "510426bca4134dd683867658c93cf839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "510426bca4134dd683867658c93cf839" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40607 } } }
I20260619 18:51:14.241691 531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:14.242131 530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "510426bca4134dd683867658c93cf839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "510426bca4134dd683867658c93cf839" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 40607 } } }
I20260619 18:51:14.242309 530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:14.242605 538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:14.245239 538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:14.250746 538 catalog_manager.cc:1347] Generated new cluster ID: 05bcf12370934ac7ad529b09c6843f71
I20260619 18:51:14.250813 538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:14.264250 538 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:14.265221 538 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:14.269831 538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839: Generated new TSK 0
I20260619 18:51:14.270663 538 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:14.277611 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.193:0
--local_ip_for_outbound_sockets=127.12.234.193
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40607
--builtin_ntp_servers=127.12.234.212:39159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 18:51:14.389694 548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:14.389967 548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:14.390024 548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:14.394156 548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:14.394239 548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:14.394322 548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.193
I20260619 18:51:14.399173 548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:39159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.193
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.193
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:14.400264 548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:14.401641 548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:14.411484 556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:14.411478 554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:14.411698 548 server_base.cc:1034] running on GCE node
W20260619 18:51:14.411478 553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:14.412205 548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:14.412871 548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:14.414027 548 hybrid_clock.cc:648] HybridClock initialized: now 1781895074414006 us; error 34 us; skew 500 ppm
I20260619 18:51:14.415910 548 webserver.cc:460] Webserver started at http://127.12.234.193:36985/ using document root <none> and password file <none>
I20260619 18:51:14.416447 548 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:14.416504 548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:14.416685 548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:14.418784 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/data/instance:
uuid: "7baf1b3f35db4db58b28d8b2ad51c97b"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "9e2a577afe815157ece8ed6d47fd0684"
server_key_iv: "efca81f13a4263a2786f101ce4042bfd"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.419301 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance:
uuid: "7baf1b3f35db4db58b28d8b2ad51c97b"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "9e2a577afe815157ece8ed6d47fd0684"
server_key_iv: "efca81f13a4263a2786f101ce4042bfd"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.423039 548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:14.425576 562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.426868 548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:51:14.426990 548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/wal
uuid: "7baf1b3f35db4db58b28d8b2ad51c97b"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "9e2a577afe815157ece8ed6d47fd0684"
server_key_iv: "efca81f13a4263a2786f101ce4042bfd"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.427095 548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:14.438094 548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:14.438836 548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:14.439009 548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:14.439601 548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:14.440549 548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:14.440598 548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.440650 548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:14.440676 548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.448901 548 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.193:42169
I20260619 18:51:14.448933 675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.193:42169 every 8 connection(s)
I20260619 18:51:14.449806 548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/data/info.pb
I20260619 18:51:14.454001 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 548
I20260619 18:51:14.454125 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-0/wal/instance
I20260619 18:51:14.454448 13227 external_mini_cluster.cc:1428] Setting key b4007d50d4ab7b7dc6c2c7476dd72cae
I20260619 18:51:14.456478 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.194:0
--local_ip_for_outbound_sockets=127.12.234.194
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40607
--builtin_ntp_servers=127.12.234.212:39159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 18:51:14.459805 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.451683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:35535 (local address 127.12.234.254:40607)
0619 18:51:14.451851 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.451855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.452865 (+ 1010us) server_negotiation.cc:407] Connection header received
0619 18:51:14.454336 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.454341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.454398 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.454493 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:14.455539 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.456158 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.457755 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.458135 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.458290 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.459347 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:14.459351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:14.459354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:14.459368 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:14.459391 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:14.459401 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:14.459468 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.459591 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.459654 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 18:51:14.460757 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.452052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40607 (local address 127.12.234.193:35535)
0619 18:51:14.452736 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:14.452776 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:14.454055 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:14.454707 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:14.454715 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:14.454808 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:14.455409 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.455423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.456293 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.456297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.457625 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.457638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.458332 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.458335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.458482 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.459075 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:51:14.459077 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:14.459122 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:51:14.459197 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:14.459478 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:14.459483 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:14.459487 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 18:51:14.459578 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 18:51:14.459817 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":112,"threads_started":1}
I20260619 18:51:14.462208 676 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40607
I20260619 18:51:14.462479 676 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:14.463165 676 heartbeater.cc:511] Master 127.12.234.254:40607 requested a full tablet report, sending...
I20260619 18:51:14.464823 488 ts_manager.cc:194] Registered new tserver with Master: 7baf1b3f35db4db58b28d8b2ad51c97b (127.12.234.193:42169)
I20260619 18:51:14.465996 488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.193:35535
W20260619 18:51:14.571064 679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:14.571337 679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:14.571375 679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:14.575490 679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:14.575551 679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:14.575606 679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.194
I20260619 18:51:14.580415 679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:39159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.194
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.194
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:14.581507 679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:14.582978 679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:14.592617 684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:14.592622 685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:14.592684 687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:14.592813 679 server_base.cc:1034] running on GCE node
I20260619 18:51:14.593295 679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:14.593921 679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:14.595086 679 hybrid_clock.cc:648] HybridClock initialized: now 1781895074595058 us; error 42 us; skew 500 ppm
I20260619 18:51:14.601284 679 webserver.cc:460] Webserver started at http://127.12.234.194:40405/ using document root <none> and password file <none>
I20260619 18:51:14.601881 679 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:14.601958 679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:14.602161 679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:14.604408 679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/data/instance:
uuid: "6b1d2afde0b7416ea185e1f291ad2545"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "05c049b7e05024e579a9aea4e4ccfebc"
server_key_iv: "6f0c7711f2474ecd3dc7d80761f10ddc"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.604954 679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance:
uuid: "6b1d2afde0b7416ea185e1f291ad2545"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "05c049b7e05024e579a9aea4e4ccfebc"
server_key_iv: "6f0c7711f2474ecd3dc7d80761f10ddc"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.608888 679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:51:14.611474 693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.612811 679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 18:51:14.612929 679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/wal
uuid: "6b1d2afde0b7416ea185e1f291ad2545"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "05c049b7e05024e579a9aea4e4ccfebc"
server_key_iv: "6f0c7711f2474ecd3dc7d80761f10ddc"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.613034 679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:14.625692 679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:14.626405 679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:14.626581 679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:14.627142 679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:14.628077 679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:14.628123 679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.628175 679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:14.628201 679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.636713 679 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.194:41841
I20260619 18:51:14.636741 806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.194:41841 every 8 connection(s)
I20260619 18:51:14.637699 679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/data/info.pb
I20260619 18:51:14.643328 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 679
I20260619 18:51:14.643469 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-1/wal/instance
I20260619 18:51:14.643743 13227 external_mini_cluster.cc:1428] Setting key 2fea639dca7a0ecf5383848ecee6d496
I20260619 18:51:14.645763 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.195:0
--local_ip_for_outbound_sockets=127.12.234.195
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40607
--builtin_ntp_servers=127.12.234.212:39159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 18:51:14.647519 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.639505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.194:47607 (local address 127.12.234.254:40607)
0619 18:51:14.639665 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.639670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.640868 (+ 1198us) server_negotiation.cc:407] Connection header received
0619 18:51:14.642328 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.642331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.642381 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.642462 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:14.643497 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.644112 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.645482 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.645698 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.645782 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.647009 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:14.647012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:14.647015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:14.647027 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:14.647049 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:14.647057 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:14.647135 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.647314 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.647379 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 18:51:14.648532 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.639975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40607 (local address 127.12.234.194:47607)
0619 18:51:14.640721 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:14.640764 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:14.642096 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:14.642651 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:14.642658 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:14.642750 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:14.643330 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.643350 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.644254 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.644258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.645364 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.645376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.645849 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.645854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.646050 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.646694 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:51:14.646696 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:14.646752 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:51:14.646834 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:14.647173 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:14.647182 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:14.647186 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 18:51:14.647299 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 18:51:14.647625 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":111,"threads_started":1}
I20260619 18:51:14.649716 807 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40607
I20260619 18:51:14.649966 807 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:14.650538 807 heartbeater.cc:511] Master 127.12.234.254:40607 requested a full tablet report, sending...
I20260619 18:51:14.651579 488 ts_manager.cc:194] Registered new tserver with Master: 6b1d2afde0b7416ea185e1f291ad2545 (127.12.234.194:41841)
I20260619 18:51:14.652137 488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.194:47607
W20260619 18:51:14.756218 810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:14.756506 810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:14.756562 810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:14.760753 810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:14.760824 810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:14.760905 810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.195
I20260619 18:51:14.765848 810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:39159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.195
--webserver_port=0
--tserver_master_addrs=127.12.234.254:40607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.195
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:14.767053 810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:14.768517 810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:14.779755 818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:14.779738 816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:14.779745 815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:14.779837 810 server_base.cc:1034] running on GCE node
I20260619 18:51:14.780429 810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:14.781114 810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:14.782253 810 hybrid_clock.cc:648] HybridClock initialized: now 1781895074782220 us; error 46 us; skew 500 ppm
I20260619 18:51:14.784139 810 webserver.cc:460] Webserver started at http://127.12.234.195:40651/ using document root <none> and password file <none>
I20260619 18:51:14.784731 810 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:14.784801 810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:14.785029 810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:14.787171 810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/data/instance:
uuid: "cf098e6354a848b7a66f89029e09207f"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "81f323bd562c6ec40d573ebc650baae4"
server_key_iv: "f5710575584aa54a989d2c1977536d7b"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.787772 810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance:
uuid: "cf098e6354a848b7a66f89029e09207f"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "81f323bd562c6ec40d573ebc650baae4"
server_key_iv: "f5710575584aa54a989d2c1977536d7b"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.791692 810 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:14.794271 824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.795553 810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:51:14.795675 810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/wal
uuid: "cf098e6354a848b7a66f89029e09207f"
format_stamp: "Formatted at 2026-06-19 18:51:14 on dist-test-slave-fljv"
server_key: "81f323bd562c6ec40d573ebc650baae4"
server_key_iv: "f5710575584aa54a989d2c1977536d7b"
server_key_version: "encryptionkey@0"
I20260619 18:51:14.795786 810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:14.819514 810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:14.820269 810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:14.820454 810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:14.821067 810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 18:51:14.822065 810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 18:51:14.822113 810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.822152 810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 18:51:14.822208 810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:14.830664 810 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.195:35073
I20260619 18:51:14.830713 937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.195:35073 every 8 connection(s)
I20260619 18:51:14.831599 810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/data/info.pb
I20260619 18:51:14.832676 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 810
I20260619 18:51:14.832789 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/wal/instance
I20260619 18:51:14.833084 13227 external_mini_cluster.cc:1428] Setting key abd909977c0644ee277d14964f2180ce
I20260619 18:51:14.840804 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.833527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.195:43261 (local address 127.12.234.254:40607)
0619 18:51:14.833654 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.833659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.834984 (+ 1325us) server_negotiation.cc:407] Connection header received
0619 18:51:14.836379 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.836381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.836425 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.836494 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:14.837391 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.837915 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.839143 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.839365 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.839445 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.840436 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:14.840438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:14.840440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:14.840450 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:14.840464 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:14.840470 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:14.840522 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.840621 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.840669 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 18:51:14.841709 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.834041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:40607 (local address 127.12.234.195:43261)
0619 18:51:14.834830 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:14.834887 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:14.836198 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:14.836628 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:14.836635 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:14.836722 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:14.837234 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.837254 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.838028 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.838031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.839024 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.839033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.839485 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.839487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.839638 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.840214 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 18:51:14.840216 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:14.840257 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 18:51:14.840323 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:14.840518 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:14.840522 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:14.840525 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 18:51:14.840619 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 18:51:14.840897 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":163,"threads_started":1}
I20260619 18:51:14.842744 938 heartbeater.cc:347] Connected to a master server at 127.12.234.254:40607
I20260619 18:51:14.842962 938 heartbeater.cc:464] Registering TS with master...
I20260619 18:51:14.843449 938 heartbeater.cc:511] Master 127.12.234.254:40607 requested a full tablet report, sending...
I20260619 18:51:14.844491 488 ts_manager.cc:194] Registered new tserver with Master: cf098e6354a848b7a66f89029e09207f (127.12.234.195:35073)
I20260619 18:51:14.845074 488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.195:43261
I20260619 18:51:14.846952 13227 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 18:51:14.852577 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.848885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39594 (local address 127.12.234.254:40607)
0619 18:51:14.849038 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.849043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.849254 (+ 211us) server_negotiation.cc:407] Connection header received
0619 18:51:14.849360 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.849363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.849407 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.849469 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:14.849765 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.850329 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.851153 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.851371 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.851456 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.852196 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:14.852200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:14.852203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:14.852216 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:14.852237 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:14.852245 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:14.852301 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.852384 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.852439 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 18:51:14.855432 488 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 18:51:14.858433 488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 18:51:14.873001 953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.867696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:42169 (local address 127.0.0.1:39072)
0619 18:51:14.867915 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:14.867926 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:14.867993 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:14.869239 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:14.869242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:14.869258 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:14.869344 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.869350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.870646 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.870649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.871692 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.871702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.872460 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.872464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.872569 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.872604 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 18:51:14.872704 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 18:51:14.872831 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":46,"thread_start_us":92,"threads_started":1}
I20260619 18:51:14.873028 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.868179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39072 (local address 127.12.234.193:42169)
0619 18:51:14.868854 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.868865 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.868889 (+ 24us) server_negotiation.cc:407] Connection header received
0619 18:51:14.868961 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.868966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.869124 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.869257 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:14.869445 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.870551 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.871802 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.872350 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.872469 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.872706 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.872775 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.872877 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":134,"threads_started":1}
I20260619 18:51:14.873368 949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.866925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:35073 (local address 127.0.0.1:58650)
0619 18:51:14.867774 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:14.867816 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:14.867955 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:14.868952 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:14.868955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:14.868974 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:14.869082 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.869090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.870552 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.870556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.871701 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.871710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.872872 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.872876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.873066 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.873087 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 18:51:14.873157 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 18:51:14.873235 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":83,"thread_start_us":97,"threads_started":1}
I20260619 18:51:14.873673 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.867179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:41841 (local address 127.0.0.1:56868)
0619 18:51:14.867772 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:14.867816 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:14.867956 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:14.868747 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:14.868752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:14.868773 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:14.868904 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.868911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.870069 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.870072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.871484 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.871495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.872878 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.872881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.873017 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.873407 (+ 390us) client_negotiation.cc:769] Sending connection context
0619 18:51:14.873484 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 18:51:14.873558 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":365,"spinlock_wait_cycles":768,"thread_start_us":105,"threads_started":1}
I20260619 18:51:14.873792 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.867079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.12.234.195:35073)
0619 18:51:14.868404 (+ 1325us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.868415 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.868438 (+ 23us) server_negotiation.cc:407] Connection header received
0619 18:51:14.868511 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.868516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.868674 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.868822 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:14.869203 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.870432 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.871819 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.872557 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.872658 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.872805 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.873542 (+ 737us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.873647 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1223,"thread_start_us":152,"threads_started":1}
I20260619 18:51:14.874233 952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.867353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56868 (local address 127.12.234.194:41841)
0619 18:51:14.867666 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.867678 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.868104 (+ 426us) server_negotiation.cc:407] Connection header received
0619 18:51:14.868175 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.868179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.868404 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.868551 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:14.869042 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.869957 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.871628 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.872425 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.872566 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.872806 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.873950 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.874060 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":117,"threads_started":1}
I20260619 18:51:14.876288 610 tablet_service.cc:1467] Processing CreateTablet for tablet 85c86bbab84649f2a67727cd5317eeeb (DEFAULT_TABLE table=test-table [id=20bd0f00b57c430ea0570c02043face4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:14.876652 872 tablet_service.cc:1467] Processing CreateTablet for tablet 85c86bbab84649f2a67727cd5317eeeb (DEFAULT_TABLE table=test-table [id=20bd0f00b57c430ea0570c02043face4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:14.876761 741 tablet_service.cc:1467] Processing CreateTablet for tablet 85c86bbab84649f2a67727cd5317eeeb (DEFAULT_TABLE table=test-table [id=20bd0f00b57c430ea0570c02043face4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 18:51:14.877310 610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c86bbab84649f2a67727cd5317eeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:14.877696 872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c86bbab84649f2a67727cd5317eeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:14.877707 741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c86bbab84649f2a67727cd5317eeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:14.884356 955 tablet_bootstrap.cc:492] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b: Bootstrap starting.
I20260619 18:51:14.884935 956 tablet_bootstrap.cc:492] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: Bootstrap starting.
I20260619 18:51:14.886505 957 tablet_bootstrap.cc:492] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545: Bootstrap starting.
I20260619 18:51:14.887200 956 tablet_bootstrap.cc:654] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:14.887200 955 tablet_bootstrap.cc:654] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:14.888093 955 log.cc:824] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:14.888093 956 log.cc:824] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:14.888374 957 tablet_bootstrap.cc:654] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:14.889089 957 log.cc:824] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:14.890367 955 tablet_bootstrap.cc:492] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b: No bootstrap required, opened a new log
I20260619 18:51:14.890568 955 ts_tablet_manager.cc:1397] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 18:51:14.891618 956 tablet_bootstrap.cc:492] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: No bootstrap required, opened a new log
I20260619 18:51:14.891815 956 ts_tablet_manager.cc:1397] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 18:51:14.891853 957 tablet_bootstrap.cc:492] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545: No bootstrap required, opened a new log
I20260619 18:51:14.892225 957 ts_tablet_manager.cc:1397] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260619 18:51:14.894908 955 raft_consensus.cc:348] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } }
I20260619 18:51:14.895215 955 raft_consensus.cc:374] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:14.895272 955 raft_consensus.cc:729] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7baf1b3f35db4db58b28d8b2ad51c97b, State: Initialized, Role: FOLLOWER
I20260619 18:51:14.895232 956 raft_consensus.cc:348] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } }
I20260619 18:51:14.895476 956 raft_consensus.cc:374] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:14.895538 956 raft_consensus.cc:729] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf098e6354a848b7a66f89029e09207f, State: Initialized, Role: FOLLOWER
I20260619 18:51:14.895848 955 consensus_queue.cc:260] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } }
I20260619 18:51:14.895923 956 consensus_queue.cc:260] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } }
I20260619 18:51:14.896203 957 raft_consensus.cc:348] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } }
I20260619 18:51:14.896440 957 raft_consensus.cc:374] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:14.896515 957 raft_consensus.cc:729] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1d2afde0b7416ea185e1f291ad2545, State: Initialized, Role: FOLLOWER
I20260619 18:51:14.896894 957 consensus_queue.cc:260] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } }
I20260619 18:51:14.896958 955 ts_tablet_manager.cc:1428] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 18:51:14.897172 956 ts_tablet_manager.cc:1428] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 18:51:14.897332 676 heartbeater.cc:503] Master 127.12.234.254:40607 was elected leader, sending a full tablet report...
I20260619 18:51:14.897526 938 heartbeater.cc:503] Master 127.12.234.254:40607 was elected leader, sending a full tablet report...
I20260619 18:51:14.897764 957 ts_tablet_manager.cc:1428] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260619 18:51:14.898038 807 heartbeater.cc:503] Master 127.12.234.254:40607 was elected leader, sending a full tablet report...
I20260619 18:51:14.930944 961 raft_consensus.cc:482] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 18:51:14.931190 961 raft_consensus.cc:504] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } }
I20260619 18:51:14.932366 961 leader_election.cc:288] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf098e6354a848b7a66f89029e09207f (127.12.234.195:35073), 6b1d2afde0b7416ea185e1f291ad2545 (127.12.234.194:41841)
I20260619 18:51:14.936236 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.932785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:44455 (local address 127.12.234.195:35073)
0619 18:51:14.932913 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.932917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.932932 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:51:14.932986 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.932989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.933043 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.933131 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:14.933446 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.934048 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.935244 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.935662 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.935763 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.935809 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.936031 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.936110 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 18:51:14.936235 952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.932765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.193:36045 (local address 127.12.234.194:41841)
0619 18:51:14.932891 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.932895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.933175 (+ 280us) server_negotiation.cc:407] Connection header received
0619 18:51:14.933287 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.933290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.933345 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.933421 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 18:51:14.933685 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.934251 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.935325 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.935736 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.935862 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.935909 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.936045 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.936115 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 18:51:14.936235 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.932575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.195:35073 (local address 127.12.234.193:44455)
0619 18:51:14.932720 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:14.932737 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:14.932831 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:14.933188 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:14.933191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:14.933214 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:14.933345 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.933352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.934148 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.934152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.935112 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.935122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.935787 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.935791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.935903 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.935927 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 18:51:14.936004 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 18:51:14.936101 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":3}
I20260619 18:51:14.936249 964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.932613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:41841 (local address 127.12.234.193:36045)
0619 18:51:14.933078 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:14.933091 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:14.933179 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:14.933435 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:14.933437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:14.933453 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 18:51:14.933576 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.933582 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.934349 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.934353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.935188 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:14.935200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.935858 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.935861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:14.935934 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.935951 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 18:51:14.936019 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 18:51:14.936112 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":126,"thread_start_us":88,"threads_started":1}
I20260619 18:51:14.936996 892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c86bbab84649f2a67727cd5317eeeb" candidate_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf098e6354a848b7a66f89029e09207f" is_pre_election: true
I20260619 18:51:14.937199 761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c86bbab84649f2a67727cd5317eeeb" candidate_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b1d2afde0b7416ea185e1f291ad2545" is_pre_election: true
I20260619 18:51:14.937361 892 raft_consensus.cc:2454] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7baf1b3f35db4db58b28d8b2ad51c97b in term 0.
I20260619 18:51:14.937441 761 raft_consensus.cc:2454] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7baf1b3f35db4db58b28d8b2ad51c97b in term 0.
I20260619 18:51:14.937914 563 leader_election.cc:302] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b1d2afde0b7416ea185e1f291ad2545, 7baf1b3f35db4db58b28d8b2ad51c97b; no voters:
I20260619 18:51:14.938242 961 raft_consensus.cc:2781] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 18:51:14.938324 961 raft_consensus.cc:482] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 18:51:14.938364 961 raft_consensus.cc:3037] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:14.939410 961 raft_consensus.cc:504] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } }
I20260619 18:51:14.939742 961 leader_election.cc:288] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [CANDIDATE]: Term 1 election: Requested vote from peers cf098e6354a848b7a66f89029e09207f (127.12.234.195:35073), 6b1d2afde0b7416ea185e1f291ad2545 (127.12.234.194:41841)
I20260619 18:51:14.940135 761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c86bbab84649f2a67727cd5317eeeb" candidate_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b1d2afde0b7416ea185e1f291ad2545"
I20260619 18:51:14.940270 892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c86bbab84649f2a67727cd5317eeeb" candidate_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf098e6354a848b7a66f89029e09207f"
I20260619 18:51:14.940260 761 raft_consensus.cc:3037] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:14.940384 892 raft_consensus.cc:3037] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:14.941628 761 raft_consensus.cc:2454] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7baf1b3f35db4db58b28d8b2ad51c97b in term 1.
I20260619 18:51:14.941628 892 raft_consensus.cc:2454] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7baf1b3f35db4db58b28d8b2ad51c97b in term 1.
I20260619 18:51:14.941920 563 leader_election.cc:302] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b1d2afde0b7416ea185e1f291ad2545, 7baf1b3f35db4db58b28d8b2ad51c97b; no voters:
I20260619 18:51:14.942083 961 raft_consensus.cc:2781] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:14.942325 961 raft_consensus.cc:686] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [term 1 LEADER]: Becoming Leader. State: Replica: 7baf1b3f35db4db58b28d8b2ad51c97b, State: Running, Role: LEADER
I20260619 18:51:14.942691 961 consensus_queue.cc:237] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } }
I20260619 18:51:14.945926 487 catalog_manager.cc:5495] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7baf1b3f35db4db58b28d8b2ad51c97b (127.12.234.193). New cstate: current_term: 1 leader_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7baf1b3f35db4db58b28d8b2ad51c97b" member_type: VOTER last_known_addr { host: "127.12.234.193" port: 42169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 } health_report { overall_health: UNKNOWN } } }
W20260619 18:51:14.951125 677 tablet.cc:2369] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:14.972954 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:14.969379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.12.234.193:42169)
0619 18:51:14.969525 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:14.969530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:14.969544 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:51:14.969631 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:14.969633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:14.969692 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:14.969782 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:14.970015 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.970607 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.971489 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:14.971703 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:14.971792 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:14.972087 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:14.972540 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:14.972657 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:14.972719 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 18:51:14.972781 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 18:51:14.978978 892 raft_consensus.cc:1261] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [term 1 FOLLOWER]: Refusing update from remote peer 7baf1b3f35db4db58b28d8b2ad51c97b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:14.978978 761 raft_consensus.cc:1261] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [term 1 FOLLOWER]: Refusing update from remote peer 7baf1b3f35db4db58b28d8b2ad51c97b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:14.979867 965 consensus_queue.cc:1035] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf098e6354a848b7a66f89029e09207f" member_type: VOTER last_known_addr { host: "127.12.234.195" port: 35073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:14.980096 961 consensus_queue.cc:1035] T 85c86bbab84649f2a67727cd5317eeeb P 7baf1b3f35db4db58b28d8b2ad51c97b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b1d2afde0b7416ea185e1f291ad2545" member_type: VOTER last_known_addr { host: "127.12.234.194" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:14.988517 970 mvcc.cc:205] Tried to move back new op lower bound from 7298642227106263040 to 7298642226968014848. Current Snapshot: MvccSnapshot[applied={T|T < 7298642227106263040}]
I20260619 18:51:14.989457 969 mvcc.cc:205] Tried to move back new op lower bound from 7298642227106263040 to 7298642226968014848. Current Snapshot: MvccSnapshot[applied={T|T < 7298642227106263040}]
I20260619 18:51:14.997994 487 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:14.998168 487 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 18:51:15.000746 487 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 510426bca4134dd683867658c93cf839: Sending DeleteTablet for 3 replicas of tablet 85c86bbab84649f2a67727cd5317eeeb
I20260619 18:51:15.001616 872 tablet_service.cc:1514] Processing DeleteTablet for tablet 85c86bbab84649f2a67727cd5317eeeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:14 UTC) from {username='slave'} at 127.0.0.1:58650
I20260619 18:51:15.001616 741 tablet_service.cc:1514] Processing DeleteTablet for tablet 85c86bbab84649f2a67727cd5317eeeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:14 UTC) from {username='slave'} at 127.0.0.1:56868
I20260619 18:51:15.002262 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 548
I20260619 18:51:15.002327 610 tablet_service.cc:1514] Processing DeleteTablet for tablet 85c86bbab84649f2a67727cd5317eeeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 18:51:14 UTC) from {username='slave'} at 127.0.0.1:39072
I20260619 18:51:15.004068 978 tablet_replica.cc:330] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545: stopping tablet replica
I20260619 18:51:15.004371 978 raft_consensus.cc:2229] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:51:15.004673 978 raft_consensus.cc:2258] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:15.005067 979 tablet_replica.cc:330] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: stopping tablet replica
I20260619 18:51:15.005395 979 raft_consensus.cc:2229] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 18:51:15.005709 979 raft_consensus.cc:2258] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 18:51:15.006521 978 ts_tablet_manager.cc:1905] T 85c86bbab84649f2a67727cd5317eeeb P 6b1d2afde0b7416ea185e1f291ad2545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 18:51:15.007277 979 ts_tablet_manager.cc:1905] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 18:51:15.008289 475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.193:42169 (error 108)
I20260619 18:51:15.009137 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 679
I20260619 18:51:15.009564 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:15.009209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.193:42169 (local address 127.0.0.1:39098)
0619 18:51:15.009413 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:15.009479 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.193:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260619 18:51:15.010064 475 catalog_manager.cc:4584] TS 7baf1b3f35db4db58b28d8b2ad51c97b (127.12.234.193:42169): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 85c86bbab84649f2a67727cd5317eeeb: Network error: Client connection negotiation failed: client connection to 127.12.234.193:42169: connect: Connection refused (error 111)
I20260619 18:51:15.010810 979 ts_tablet_manager.cc:1918] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 18:51:15.010900 979 log.cc:1196] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: Deleting WAL directory at /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781894826428600-13227-0/minicluster-data/ts-2/wal/wals/85c86bbab84649f2a67727cd5317eeeb
I20260619 18:51:15.011191 979 ts_tablet_manager.cc:1939] T 85c86bbab84649f2a67727cd5317eeeb P cf098e6354a848b7a66f89029e09207f: Deleting consensus metadata
I20260619 18:51:15.012372 475 catalog_manager.cc:4841] TS cf098e6354a848b7a66f89029e09207f (127.12.234.195:35073): tablet 85c86bbab84649f2a67727cd5317eeeb (table test-table [id=20bd0f00b57c430ea0570c02043face4]) successfully deleted
I20260619 18:51:15.014735 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 810
I20260619 18:51:15.018482 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:15.014894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.194:41841 (local address 127.0.0.1:56874)
0619 18:51:15.018334 (+ 3440us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:15.018396 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.194:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3352}
I20260619 18:51:15.019954 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 457
2026-06-19T18:51:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1010 ms)
[----------] 6 tests from EncryptionPolicyTest (10626 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:15 dist-test-slave-fljv krb5kdc[983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:15 dist-test-slave-fljv krb5kdc[983](info): set up 2 sockets
Jun 19 18:51:15 dist-test-slave-fljv krb5kdc[983](info): commencing operation
krb5kdc: starting...
W20260619 18:51:15.982283 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:16 dist-test-slave-fljv krb5kdc[983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:16.105139 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:16.215159 996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:16.215449 996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:16.215490 996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:16.219937 996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:16.220009 996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:16.220028 996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:16.220050 996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:16.220068 996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:16.225567 996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46075
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:16.226783 996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:16.227828 996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:16.236294 1002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:16.236378 996 server_base.cc:1034] running on GCE node
W20260619 18:51:16.236297 1004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:16.236289 1001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:16.237255 996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:16.238464 996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:16.239621 996 hybrid_clock.cc:648] HybridClock initialized: now 1781895076239606 us; error 30 us; skew 500 ppm
Jun 19 18:51:16 dist-test-slave-fljv krb5kdc[983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895076, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:16.242439 996 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:16.243436 996 webserver.cc:460] Webserver started at http://127.0.0.1:44741/ using document root <none> and password file <none>
I20260619 18:51:16.243997 996 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:16.244063 996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:16.244270 996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:16.246356 996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "950880f5b741424a80eb4527c23f7a2a"
format_stamp: "Formatted at 2026-06-19 18:51:16 on dist-test-slave-fljv"
server_key: "81c35b3d6cde2b626b23f7c3ac6fb758"
server_key_iv: "0a04ca86b492f215d3cf4346eb916b8f"
server_key_version: "encryptionkey@0"
I20260619 18:51:16.246894 996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "950880f5b741424a80eb4527c23f7a2a"
format_stamp: "Formatted at 2026-06-19 18:51:16 on dist-test-slave-fljv"
server_key: "81c35b3d6cde2b626b23f7c3ac6fb758"
server_key_iv: "0a04ca86b492f215d3cf4346eb916b8f"
server_key_version: "encryptionkey@0"
I20260619 18:51:16.250751 996 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 18:51:16.253350 1011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:16.254763 996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:16.254884 996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "950880f5b741424a80eb4527c23f7a2a"
format_stamp: "Formatted at 2026-06-19 18:51:16 on dist-test-slave-fljv"
server_key: "81c35b3d6cde2b626b23f7c3ac6fb758"
server_key_iv: "0a04ca86b492f215d3cf4346eb916b8f"
server_key_version: "encryptionkey@0"
I20260619 18:51:16.254988 996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:16.273686 996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:16.276924 996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:16.277115 996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:16.284392 996 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46075
I20260619 18:51:16.284422 1063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46075 every 8 connection(s)
I20260619 18:51:16.285378 996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:16.288297 1064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:16.291504 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 996
I20260619 18:51:16.291627 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:16.291926 13227 external_mini_cluster.cc:1428] Setting key abe9711746f401484109dde986459d72
I20260619 18:51:16.294307 1064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a: Bootstrap starting.
I20260619 18:51:16.296828 1064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:16.297667 1064 log.cc:824] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:16.299849 1064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a: No bootstrap required, opened a new log
Jun 19 18:51:16 dist-test-slave-fljv krb5kdc[983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 18:51:16.303151 1064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950880f5b741424a80eb4527c23f7a2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46075 } }
I20260619 18:51:16.303449 1064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:16.303521 1064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 950880f5b741424a80eb4527c23f7a2a, State: Initialized, Role: FOLLOWER
I20260619 18:51:16.303972 1064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950880f5b741424a80eb4527c23f7a2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46075 } }
I20260619 18:51:16.304106 1064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:16.304175 1064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:16.304280 1064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:16.305393 1064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950880f5b741424a80eb4527c23f7a2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46075 } }
I20260619 18:51:16.305740 1064 leader_election.cc:302] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 950880f5b741424a80eb4527c23f7a2a; no voters:
I20260619 18:51:16.306042 1064 leader_election.cc:288] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:16.306139 1069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:16.306331 1069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [term 1 LEADER]: Becoming Leader. State: Replica: 950880f5b741424a80eb4527c23f7a2a, State: Running, Role: LEADER
I20260619 18:51:16.306648 1069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950880f5b741424a80eb4527c23f7a2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46075 } }
I20260619 18:51:16.307430 1064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:16.307458 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:16.293414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38985 (local address 127.0.0.1:46075)
0619 18:51:16.293828 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:16.293843 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:16.293880 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:51:16.294594 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:16.294615 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:16.294967 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:16.295264 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:16.296172 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:16.297213 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:16.298369 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:16.298865 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:16.299034 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:16.301662 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:16.301668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:16.301682 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:16.301710 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:16.304047 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:16.304648 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:16.304655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:16.304660 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:16.304784 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:16.305093 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:16.305096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:16.305097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:16.305532 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:16.305731 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:16.306057 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 18:51:16.306652 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":126,"threads_started":1}
I20260619 18:51:16.307857 1069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 950880f5b741424a80eb4527c23f7a2a. Latest consensus state: current_term: 1 leader_uuid: "950880f5b741424a80eb4527c23f7a2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950880f5b741424a80eb4527c23f7a2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46075 } } }
I20260619 18:51:16.307963 1069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:16.308239 1070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "950880f5b741424a80eb4527c23f7a2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950880f5b741424a80eb4527c23f7a2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46075 } } }
I20260619 18:51:16.308326 1070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:16.310973 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 18:51:16.312645 1084 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:51:16.312726 1084 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:16.312793 1073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:16.313685 1073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:16.319638 1073 catalog_manager.cc:1347] Generated new cluster ID: 0f8cd27d06044ae9b966bc2bb9495c64
I20260619 18:51:16.319710 1073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:16.324565 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:16.313146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45179 (local address 127.0.0.1:46075)
0619 18:51:16.313253 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:16.313256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:16.313509 (+ 253us) server_negotiation.cc:407] Connection header received
0619 18:51:16.313644 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:16.313646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:16.313681 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:16.313750 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:16.314605 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:16.315384 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:16.316472 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:16.316824 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:16.316942 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:16.318527 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:16.318531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:16.318534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:16.318560 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:16.322771 (+ 4211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:16.323284 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:16.323289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:16.323290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:16.323369 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:16.323685 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:16.323688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:16.323690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:16.323866 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:16.323994 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:16.324299 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 18:51:16.324391 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 18:51:16.348443 1073 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:16.349808 1073 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:16.357045 1073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 950880f5b741424a80eb4527c23f7a2a: Generated new TSK 0
I20260619 18:51:16.358008 1073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:16.364362 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 996
W20260619 18:51:16.373387 981 connection.cc:535] client connection to 127.0.0.1:46075 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46075): Connection reset by peer (error 104)
2026-06-19T18:51:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:16 dist-test-slave-fljv krb5kdc[1095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:16 dist-test-slave-fljv krb5kdc[1095](info): set up 2 sockets
Jun 19 18:51:16 dist-test-slave-fljv krb5kdc[1095](info): commencing operation
krb5kdc: starting...
W20260619 18:51:17.326277 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:17 dist-test-slave-fljv krb5kdc[1095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:17.445267 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36265
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:17.557331 1108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:17.557646 1108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:17.557690 1108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:17.561919 1108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:17.561987 1108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:17.562006 1108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:17.562022 1108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:17.562036 1108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:17.567530 1108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36265
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:17.568656 1108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:17.569697 1108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:17.578037 1114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:17.578064 1113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:17.578204 1108 server_base.cc:1034] running on GCE node
W20260619 18:51:17.578069 1116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:17.578895 1108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:17.580016 1108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:17.581211 1108 hybrid_clock.cc:648] HybridClock initialized: now 1781895077581192 us; error 31 us; skew 500 ppm
Jun 19 18:51:17 dist-test-slave-fljv krb5kdc[1095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895077, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:17.584018 1108 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:17.584981 1108 webserver.cc:460] Webserver started at http://127.0.0.1:33907/ using document root <none> and password file <none>
I20260619 18:51:17.585539 1108 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:17.585599 1108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:17.585776 1108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:17.587844 1108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "7a985d7bde2f4556a646a16ee4d7c816"
format_stamp: "Formatted at 2026-06-19 18:51:17 on dist-test-slave-fljv"
server_key: "6c1f79bfe78ed0b1771662f394e87956"
server_key_iv: "10b3e18c48ad39c506b4e7f8a47dc364"
server_key_version: "encryptionkey@0"
I20260619 18:51:17.588359 1108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "7a985d7bde2f4556a646a16ee4d7c816"
format_stamp: "Formatted at 2026-06-19 18:51:17 on dist-test-slave-fljv"
server_key: "6c1f79bfe78ed0b1771662f394e87956"
server_key_iv: "10b3e18c48ad39c506b4e7f8a47dc364"
server_key_version: "encryptionkey@0"
I20260619 18:51:17.592095 1108 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:51:17.594633 1123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:17.595862 1108 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:51:17.595970 1108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "7a985d7bde2f4556a646a16ee4d7c816"
format_stamp: "Formatted at 2026-06-19 18:51:17 on dist-test-slave-fljv"
server_key: "6c1f79bfe78ed0b1771662f394e87956"
server_key_iv: "10b3e18c48ad39c506b4e7f8a47dc364"
server_key_version: "encryptionkey@0"
I20260619 18:51:17.596063 1108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:17.623706 1108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:17.626968 1108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:17.627149 1108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:17.634404 1108 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36265
I20260619 18:51:17.634424 1175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36265 every 8 connection(s)
I20260619 18:51:17.635450 1108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:17.638350 1176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:17.641827 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 1108
I20260619 18:51:17.641968 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:17.642326 13227 external_mini_cluster.cc:1428] Setting key 46355395cda4fa9b5d3c48d9bec2537c
I20260619 18:51:17.644739 1176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816: Bootstrap starting.
I20260619 18:51:17.647171 1176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:17.648105 1176 log.cc:824] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:17.650364 1176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816: No bootstrap required, opened a new log
I20260619 18:51:17.651690 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:17.643852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34483 (local address 127.0.0.1:36265)
0619 18:51:17.644283 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:17.644298 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:17.644333 (+ 35us) server_negotiation.cc:407] Connection header received
0619 18:51:17.645028 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:17.645048 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:17.645416 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:17.645702 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:17.646069 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:17.647130 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:17.648338 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:17.648842 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:17.648963 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:17.649851 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:17.649857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:17.649870 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:17.650374 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:17.650444 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:17.650478 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:17.650586 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:17.650766 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 18:51:17.650902 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
I20260619 18:51:17.654510 1176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a985d7bde2f4556a646a16ee4d7c816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36265 } }
I20260619 18:51:17.654817 1176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:17.654881 1176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a985d7bde2f4556a646a16ee4d7c816, State: Initialized, Role: FOLLOWER
I20260619 18:51:17.655015 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:17.655359 1176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a985d7bde2f4556a646a16ee4d7c816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36265 } }
I20260619 18:51:17.655541 1176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:17.655607 1176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:17.655704 1176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:17.656956 1176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a985d7bde2f4556a646a16ee4d7c816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36265 } }
I20260619 18:51:17.657346 1176 leader_election.cc:302] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a985d7bde2f4556a646a16ee4d7c816; no voters:
I20260619 18:51:17.657696 1176 leader_election.cc:288] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:17.657792 1186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:17.658006 1186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [term 1 LEADER]: Becoming Leader. State: Replica: 7a985d7bde2f4556a646a16ee4d7c816, State: Running, Role: LEADER
I20260619 18:51:17.658385 1186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a985d7bde2f4556a646a16ee4d7c816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36265 } }
I20260619 18:51:17.659126 1176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:17.659917 1188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a985d7bde2f4556a646a16ee4d7c816" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a985d7bde2f4556a646a16ee4d7c816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36265 } } }
I20260619 18:51:17.660043 1188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:17.660279 1188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a985d7bde2f4556a646a16ee4d7c816. Latest consensus state: current_term: 1 leader_uuid: "7a985d7bde2f4556a646a16ee4d7c816" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a985d7bde2f4556a646a16ee4d7c816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36265 } } }
I20260619 18:51:17.660358 1188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:17.660625 1195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:17.662202 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:17.657299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38555 (local address 127.0.0.1:36265)
0619 18:51:17.657428 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:17.657432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:17.657628 (+ 196us) server_negotiation.cc:407] Connection header received
0619 18:51:17.657748 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:17.657751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:17.657799 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:17.657885 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:17.658253 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:17.658975 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:17.660165 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:17.660489 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:17.660578 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:17.661655 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:17.661658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:17.661661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:17.661674 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:17.661703 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:17.661716 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:17.661798 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:17.662014 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 18:51:17.662050 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:51:17.664285 1195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:17.670455 1195 catalog_manager.cc:1347] Generated new cluster ID: be5046066a164b0394bc2d6dc675039b
I20260619 18:51:17.670550 1195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:17.700800 1195 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:17.701800 1195 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:17.709295 1195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a985d7bde2f4556a646a16ee4d7c816: Generated new TSK 0
I20260619 18:51:17.710053 1195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:17.718326 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 1108
W20260619 18:51:17.722923 1093 connection.cc:535] client connection to 127.0.0.1:36265 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36265): Connection reset by peer (error 104)
2026-06-19T18:51:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:17 dist-test-slave-fljv krb5kdc[1207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:17 dist-test-slave-fljv krb5kdc[1207](info): set up 2 sockets
Jun 19 18:51:17 dist-test-slave-fljv krb5kdc[1207](info): commencing operation
krb5kdc: starting...
W20260619 18:51:18.686285 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:18 dist-test-slave-fljv krb5kdc[1207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:18.805833 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:18.915947 1220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:18.916234 1220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:18.916275 1220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:18.920295 1220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:18.920370 1220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:18.920390 1220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:18.920406 1220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:18.920420 1220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:18.925879 1220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:18.927248 1220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:18.928325 1220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:18.937085 1226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:18.937069 1228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:18.937223 1220 server_base.cc:1034] running on GCE node
W20260619 18:51:18.937069 1225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:18.938009 1220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:18.939285 1220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:18.940471 1220 hybrid_clock.cc:648] HybridClock initialized: now 1781895078940456 us; error 32 us; skew 500 ppm
Jun 19 18:51:18 dist-test-slave-fljv krb5kdc[1207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895078, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:18.943318 1220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:18.944309 1220 webserver.cc:460] Webserver started at http://127.0.0.1:34757/ using document root <none> and password file <none>
I20260619 18:51:18.944877 1220 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:18.944952 1220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:18.945135 1220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:18.947511 1220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "d39c81abc6234d68b24fca34e9c688e5"
format_stamp: "Formatted at 2026-06-19 18:51:18 on dist-test-slave-fljv"
server_key: "9992aa88c3a34a585d753f5b7e2fa68a"
server_key_iv: "543a550191b9f04bc7d8c66d3eb3f34e"
server_key_version: "encryptionkey@0"
I20260619 18:51:18.948118 1220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "d39c81abc6234d68b24fca34e9c688e5"
format_stamp: "Formatted at 2026-06-19 18:51:18 on dist-test-slave-fljv"
server_key: "9992aa88c3a34a585d753f5b7e2fa68a"
server_key_iv: "543a550191b9f04bc7d8c66d3eb3f34e"
server_key_version: "encryptionkey@0"
I20260619 18:51:18.952189 1220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:51:18.954741 1235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:18.956139 1220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:18.956259 1220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "d39c81abc6234d68b24fca34e9c688e5"
format_stamp: "Formatted at 2026-06-19 18:51:18 on dist-test-slave-fljv"
server_key: "9992aa88c3a34a585d753f5b7e2fa68a"
server_key_iv: "543a550191b9f04bc7d8c66d3eb3f34e"
server_key_version: "encryptionkey@0"
I20260619 18:51:18.956360 1220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:18.961126 1220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:18.964313 1220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:18.964461 1220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:18.970980 1220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34669
I20260619 18:51:18.971082 1287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34669 every 8 connection(s)
I20260619 18:51:18.971891 1220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:18.974696 1288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:18.980178 1288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5: Bootstrap starting.
I20260619 18:51:18.981842 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 1220
I20260619 18:51:18.981957 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:18.982285 13227 external_mini_cluster.cc:1428] Setting key b3b880a2e9896072775f157154058ca0
I20260619 18:51:18.982805 1288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:18.983623 1288 log.cc:824] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:18.985567 1288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5: No bootstrap required, opened a new log
I20260619 18:51:18.987591 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:18.983436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37107 (local address 127.0.0.1:34669)
0619 18:51:18.983939 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:18.983967 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:18.984013 (+ 46us) server_negotiation.cc:407] Connection header received
0619 18:51:18.984774 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:18.984795 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:18.985120 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:18.985423 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:18.986262 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:18.986268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:18.986281 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:18.986290 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:18.986340 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:18.986359 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:18.986445 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:18.986622 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 18:51:18.986789 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":134,"threads_started":1}
I20260619 18:51:18.989504 1288 raft_consensus.cc:348] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39c81abc6234d68b24fca34e9c688e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260619 18:51:18.989774 1288 raft_consensus.cc:374] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:18.989848 1288 raft_consensus.cc:729] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d39c81abc6234d68b24fca34e9c688e5, State: Initialized, Role: FOLLOWER
I20260619 18:51:18.990331 1288 consensus_queue.cc:260] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39c81abc6234d68b24fca34e9c688e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260619 18:51:18.990492 1288 raft_consensus.cc:388] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:18.990485 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:18.990559 1288 raft_consensus.cc:482] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:18.990655 1288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:18.991588 1288 raft_consensus.cc:504] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39c81abc6234d68b24fca34e9c688e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260619 18:51:18.991943 1288 leader_election.cc:302] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d39c81abc6234d68b24fca34e9c688e5; no voters:
I20260619 18:51:18.992240 1288 leader_election.cc:288] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:18.992396 1297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:18.992683 1297 raft_consensus.cc:686] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [term 1 LEADER]: Becoming Leader. State: Replica: d39c81abc6234d68b24fca34e9c688e5, State: Running, Role: LEADER
I20260619 18:51:18.992983 1297 consensus_queue.cc:237] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39c81abc6234d68b24fca34e9c688e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260619 18:51:18.993655 1288 sys_catalog.cc:564] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:18.994314 1297 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d39c81abc6234d68b24fca34e9c688e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39c81abc6234d68b24fca34e9c688e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } } }
I20260619 18:51:18.994445 1297 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:18.994387 1298 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d39c81abc6234d68b24fca34e9c688e5. Latest consensus state: current_term: 1 leader_uuid: "d39c81abc6234d68b24fca34e9c688e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39c81abc6234d68b24fca34e9c688e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } } }
I20260619 18:51:18.994668 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:18.992567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36605 (local address 127.0.0.1:34669)
0619 18:51:18.992686 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:18.992691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:18.993085 (+ 394us) server_negotiation.cc:407] Connection header received
0619 18:51:18.993235 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:18.993238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:18.993280 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:18.993374 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:18.994135 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:18.994139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:18.994142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:18.994161 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:18.994219 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:18.994234 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:18.994320 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:18.994499 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 18:51:18.994545 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 18:51:18.994843 1305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:18.995414 1298 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:18.998718 1305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:19.004446 1305 catalog_manager.cc:1347] Generated new cluster ID: bff3e4bef8c0434dbaea87e50f0c14e6
I20260619 18:51:19.004525 1305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:19.025003 1305 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:19.025333 1305 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:19.031767 1305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d39c81abc6234d68b24fca34e9c688e5: Generated new TSK 0
I20260619 18:51:19.032567 1305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:19.038491 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 1220
2026-06-19T18:51:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:19 dist-test-slave-fljv krb5kdc[1319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:19 dist-test-slave-fljv krb5kdc[1319](info): set up 2 sockets
Jun 19 18:51:19 dist-test-slave-fljv krb5kdc[1319](info): commencing operation
krb5kdc: starting...
W20260619 18:51:19.998246 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:20 dist-test-slave-fljv krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:20.119645 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39081
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:20.232405 1332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:20.232715 1332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:20.232775 1332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:20.236935 1332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:20.237018 1332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:20.237046 1332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:20.237068 1332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:20.237097 1332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:20.242650 1332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:20.243774 1332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:20.244856 1332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:51:20.253731 1332 server_base.cc:1034] running on GCE node
W20260619 18:51:20.253713 1337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:20.253681 1340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:20.253681 1338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:20.254720 1332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:20.255874 1332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:20.257077 1332 hybrid_clock.cc:648] HybridClock initialized: now 1781895080257050 us; error 38 us; skew 500 ppm
Jun 19 18:51:20 dist-test-slave-fljv krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895080, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:20.265790 1332 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:20.266886 1332 webserver.cc:460] Webserver started at http://127.0.0.1:45353/ using document root <none> and password file <none>
I20260619 18:51:20.267549 1332 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:20.267619 1332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:20.267817 1332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:20.269979 1332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "d0bed3d054694a7882a5a4d8fc7c09e0"
format_stamp: "Formatted at 2026-06-19 18:51:20 on dist-test-slave-fljv"
server_key: "c9981a144bb14f589447d671bc32d339"
server_key_iv: "498d27337e3b6e46b0fadf532c03ea18"
server_key_version: "encryptionkey@0"
I20260619 18:51:20.270603 1332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "d0bed3d054694a7882a5a4d8fc7c09e0"
format_stamp: "Formatted at 2026-06-19 18:51:20 on dist-test-slave-fljv"
server_key: "c9981a144bb14f589447d671bc32d339"
server_key_iv: "498d27337e3b6e46b0fadf532c03ea18"
server_key_version: "encryptionkey@0"
I20260619 18:51:20.274603 1332 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:20.277302 1347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:20.278777 1332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:20.278916 1332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "d0bed3d054694a7882a5a4d8fc7c09e0"
format_stamp: "Formatted at 2026-06-19 18:51:20 on dist-test-slave-fljv"
server_key: "c9981a144bb14f589447d671bc32d339"
server_key_iv: "498d27337e3b6e46b0fadf532c03ea18"
server_key_version: "encryptionkey@0"
I20260619 18:51:20.279042 1332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:20.291481 1332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:20.294880 1332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:20.295111 1332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:20.301972 1332 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39081
I20260619 18:51:20.302012 1399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39081 every 8 connection(s)
I20260619 18:51:20.302999 1332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:20.305941 1400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:20.306488 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 1332
I20260619 18:51:20.306619 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:20.306879 13227 external_mini_cluster.cc:1428] Setting key e3b2303e619b6572be6dfc5b9618f913
I20260619 18:51:20.313537 1400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0: Bootstrap starting.
I20260619 18:51:20.316397 1400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0: Neither blocks nor log segments found. Creating new log.
Jun 19 18:51:20 dist-test-slave-fljv krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 18:51:20.317183 1400 log.cc:824] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:20.319796 1400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0: No bootstrap required, opened a new log
I20260619 18:51:20.322858 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:20.308354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60783 (local address 127.0.0.1:39081)
0619 18:51:20.308829 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:20.308848 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:20.308893 (+ 45us) server_negotiation.cc:407] Connection header received
0619 18:51:20.309659 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:20.309684 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:20.310207 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:20.310593 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:20.311332 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:20.312859 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:20.313787 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:20.314384 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:20.314507 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:20.317321 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:20.317327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:20.317346 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:20.317382 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:20.319332 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:20.319803 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:20.319810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:20.319816 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:20.319967 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:20.320336 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:20.320339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:20.320341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:20.320906 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:20.321102 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:20.321402 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 18:51:20.321749 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":118,"threads_started":1}
I20260619 18:51:20.324276 1400 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bed3d054694a7882a5a4d8fc7c09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39081 } }
I20260619 18:51:20.324576 1400 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:20.324637 1400 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0bed3d054694a7882a5a4d8fc7c09e0, State: Initialized, Role: FOLLOWER
I20260619 18:51:20.325114 1400 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bed3d054694a7882a5a4d8fc7c09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39081 } }
I20260619 18:51:20.325263 1400 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:20.325333 1400 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:20.325436 1400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:20.326155 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:20.326642 1400 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bed3d054694a7882a5a4d8fc7c09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39081 } }
I20260619 18:51:20.326987 1400 leader_election.cc:302] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0bed3d054694a7882a5a4d8fc7c09e0; no voters:
I20260619 18:51:20.327294 1400 leader_election.cc:288] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:20.327416 1405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:20.327654 1405 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [term 1 LEADER]: Becoming Leader. State: Replica: d0bed3d054694a7882a5a4d8fc7c09e0, State: Running, Role: LEADER
I20260619 18:51:20.327984 1405 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bed3d054694a7882a5a4d8fc7c09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39081 } }
I20260619 18:51:20.328918 1400 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:20.329389 1405 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0bed3d054694a7882a5a4d8fc7c09e0. Latest consensus state: current_term: 1 leader_uuid: "d0bed3d054694a7882a5a4d8fc7c09e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bed3d054694a7882a5a4d8fc7c09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39081 } } }
I20260619 18:51:20.329520 1405 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:20.329799 1410 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0bed3d054694a7882a5a4d8fc7c09e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bed3d054694a7882a5a4d8fc7c09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39081 } } }
I20260619 18:51:20.329927 1410 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0 [sys.catalog]: This master's current role is: LEADER
W20260619 18:51:20.333837 1425 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:51:20.333920 1425 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:20.334000 1414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:20.334918 1414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:20.336917 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:20.328429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40477 (local address 127.0.0.1:39081)
0619 18:51:20.328541 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:20.328546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:20.328858 (+ 312us) server_negotiation.cc:407] Connection header received
0619 18:51:20.328981 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:20.328984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:20.329037 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:20.329128 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:20.330149 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:20.330918 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:20.331967 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:20.332209 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:20.332303 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:20.333689 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:20.333693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:20.333696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:20.333723 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:20.335377 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:20.335795 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:20.335799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:20.335800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:20.335876 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:20.336121 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:20.336124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:20.336126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:20.336316 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:20.336427 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:20.336653 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 18:51:20.336767 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:51:20.341240 1414 catalog_manager.cc:1347] Generated new cluster ID: 4e139119fd6f41bcb2d279451c73e03d
I20260619 18:51:20.341316 1414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:20.350755 1414 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:20.351913 1414 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:20.358690 1414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0bed3d054694a7882a5a4d8fc7c09e0: Generated new TSK 0
I20260619 18:51:20.359578 1414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:20.371560 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 1332
2026-06-19T18:51:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:20 dist-test-slave-fljv krb5kdc[1431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:20 dist-test-slave-fljv krb5kdc[1431](info): set up 2 sockets
Jun 19 18:51:20 dist-test-slave-fljv krb5kdc[1431](info): commencing operation
krb5kdc: starting...
W20260619 18:51:21.334117 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:21 dist-test-slave-fljv krb5kdc[1431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:21.453719 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44947
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:21.564692 1444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:21.564977 1444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:21.565019 1444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:21.569093 1444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:21.569160 1444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:21.569180 1444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:21.569195 1444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:21.569208 1444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:21.574816 1444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:21.575897 1444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:21.576926 1444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:21.585386 1452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:21.585458 1450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:21.585386 1449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:21.585599 1444 server_base.cc:1034] running on GCE node
I20260619 18:51:21.586359 1444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:21.587530 1444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:21.588716 1444 hybrid_clock.cc:648] HybridClock initialized: now 1781895081588693 us; error 38 us; skew 500 ppm
Jun 19 18:51:21 dist-test-slave-fljv krb5kdc[1431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895081, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:21.591672 1444 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:21.592654 1444 webserver.cc:460] Webserver started at http://127.0.0.1:38803/ using document root <none> and password file <none>
I20260619 18:51:21.593214 1444 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:21.593281 1444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:21.593469 1444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:21.595604 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "dc15f0972ffe4ec2ad505fff7d848932"
format_stamp: "Formatted at 2026-06-19 18:51:21 on dist-test-slave-fljv"
server_key: "19ce38d7c3718aa9fed0124a95f2a017"
server_key_iv: "39c07372ec314c4c698c08b5763f142f"
server_key_version: "encryptionkey@0"
I20260619 18:51:21.596212 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "dc15f0972ffe4ec2ad505fff7d848932"
format_stamp: "Formatted at 2026-06-19 18:51:21 on dist-test-slave-fljv"
server_key: "19ce38d7c3718aa9fed0124a95f2a017"
server_key_iv: "39c07372ec314c4c698c08b5763f142f"
server_key_version: "encryptionkey@0"
I20260619 18:51:21.600087 1444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260619 18:51:21.602676 1459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:21.604010 1444 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:51:21.604131 1444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "dc15f0972ffe4ec2ad505fff7d848932"
format_stamp: "Formatted at 2026-06-19 18:51:21 on dist-test-slave-fljv"
server_key: "19ce38d7c3718aa9fed0124a95f2a017"
server_key_iv: "39c07372ec314c4c698c08b5763f142f"
server_key_version: "encryptionkey@0"
I20260619 18:51:21.604239 1444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:21.634539 1444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:21.637553 1444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:21.637746 1444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:21.644491 1444 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44947
I20260619 18:51:21.644587 1511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44947 every 8 connection(s)
I20260619 18:51:21.645538 1444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:21.648332 1512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:21.650338 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 1444
I20260619 18:51:21.650468 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:21.650743 13227 external_mini_cluster.cc:1428] Setting key 33e412fde95ba083d4fa3860bfd88a3d
I20260619 18:51:21.654492 1512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932: Bootstrap starting.
I20260619 18:51:21.657056 1512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:21.657881 1512 log.cc:824] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:21.660118 1512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932: No bootstrap required, opened a new log
I20260619 18:51:21.660300 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:21.652136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54869 (local address 127.0.0.1:44947)
0619 18:51:21.652544 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:21.652562 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:21.652596 (+ 34us) server_negotiation.cc:407] Connection header received
0619 18:51:21.653592 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:21.653613 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:21.654034 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:21.654400 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:21.654790 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:21.655769 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:21.656964 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:21.657470 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:21.657598 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:21.658451 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:21.658457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:21.658475 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:21.658930 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:21.658997 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:21.659023 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:21.659098 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:21.659272 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 18:51:21.659471 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":115,"threads_started":1}
I20260619 18:51:21.663120 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:21.663971 1512 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc15f0972ffe4ec2ad505fff7d848932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44947 } }
I20260619 18:51:21.664258 1512 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:21.664320 1512 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc15f0972ffe4ec2ad505fff7d848932, State: Initialized, Role: FOLLOWER
I20260619 18:51:21.664760 1512 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc15f0972ffe4ec2ad505fff7d848932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44947 } }
I20260619 18:51:21.664920 1512 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:21.664991 1512 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:21.665108 1512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:21.665997 1512 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc15f0972ffe4ec2ad505fff7d848932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44947 } }
I20260619 18:51:21.666361 1512 leader_election.cc:302] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc15f0972ffe4ec2ad505fff7d848932; no voters:
I20260619 18:51:21.666647 1512 leader_election.cc:288] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:21.666724 1522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:21.666920 1522 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [term 1 LEADER]: Becoming Leader. State: Replica: dc15f0972ffe4ec2ad505fff7d848932, State: Running, Role: LEADER
I20260619 18:51:21.667274 1522 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc15f0972ffe4ec2ad505fff7d848932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44947 } }
I20260619 18:51:21.668008 1512 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:21.668996 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:21.665016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44949 (local address 127.0.0.1:44947)
0619 18:51:21.665130 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:21.665135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:21.665324 (+ 189us) server_negotiation.cc:407] Connection header received
0619 18:51:21.665430 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:21.665433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:21.665476 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:21.665537 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:21.665787 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:21.666376 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:21.667336 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:21.667592 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:21.667686 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:21.668478 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:21.668482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:21.668484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:21.668499 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:21.668537 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:21.668551 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:21.668622 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:21.668796 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 18:51:21.668858 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 18:51:21.668967 1524 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc15f0972ffe4ec2ad505fff7d848932. Latest consensus state: current_term: 1 leader_uuid: "dc15f0972ffe4ec2ad505fff7d848932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc15f0972ffe4ec2ad505fff7d848932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44947 } } }
I20260619 18:51:21.669091 1524 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:21.669349 1523 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc15f0972ffe4ec2ad505fff7d848932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc15f0972ffe4ec2ad505fff7d848932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44947 } } }
I20260619 18:51:21.669440 1523 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:21.670066 1531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:21.672807 1531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:21.678040 1531 catalog_manager.cc:1347] Generated new cluster ID: 04fc5e8567fb493aba649898579126a8
I20260619 18:51:21.678120 1531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:21.698490 1531 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:21.699481 1531 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:21.704722 1531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc15f0972ffe4ec2ad505fff7d848932: Generated new TSK 0
I20260619 18:51:21.705428 1531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:21.719429 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 1444
2026-06-19T18:51:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:21 dist-test-slave-fljv krb5kdc[1543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:21 dist-test-slave-fljv krb5kdc[1543](info): set up 2 sockets
Jun 19 18:51:21 dist-test-slave-fljv krb5kdc[1543](info): commencing operation
krb5kdc: starting...
W20260619 18:51:22.686148 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:22 dist-test-slave-fljv krb5kdc[1543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:22.809218 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:22.922390 1556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:22.922684 1556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:22.922729 1556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:22.927037 1556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:22.927112 1556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:22.927131 1556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:22.927146 1556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:22.927176 1556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:22.932756 1556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:22.933835 1556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:22.934947 1556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:22.943719 1561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:22.943719 1562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:22.943881 1556 server_base.cc:1034] running on GCE node
W20260619 18:51:22.944068 1564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:22.944679 1556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:22.945793 1556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:22.946965 1556 hybrid_clock.cc:648] HybridClock initialized: now 1781895082946945 us; error 40 us; skew 500 ppm
Jun 19 18:51:22 dist-test-slave-fljv krb5kdc[1543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895082, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:22.949741 1556 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:22.950798 1556 webserver.cc:460] Webserver started at http://127.0.0.1:42955/ using document root <none> and password file <none>
I20260619 18:51:22.951360 1556 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:22.951429 1556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:22.951655 1556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:22.953874 1556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "ec155618bd124af090642ea62b43d390"
format_stamp: "Formatted at 2026-06-19 18:51:22 on dist-test-slave-fljv"
server_key: "08569b29216b7dd1f37e6c849492be6b"
server_key_iv: "77b23e2aeba089f7550208e218f3f182"
server_key_version: "encryptionkey@0"
I20260619 18:51:22.954598 1556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "ec155618bd124af090642ea62b43d390"
format_stamp: "Formatted at 2026-06-19 18:51:22 on dist-test-slave-fljv"
server_key: "08569b29216b7dd1f37e6c849492be6b"
server_key_iv: "77b23e2aeba089f7550208e218f3f182"
server_key_version: "encryptionkey@0"
I20260619 18:51:22.958518 1556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:22.961016 1571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:22.962479 1556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:22.962615 1556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "ec155618bd124af090642ea62b43d390"
format_stamp: "Formatted at 2026-06-19 18:51:22 on dist-test-slave-fljv"
server_key: "08569b29216b7dd1f37e6c849492be6b"
server_key_iv: "77b23e2aeba089f7550208e218f3f182"
server_key_version: "encryptionkey@0"
I20260619 18:51:22.962718 1556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:22.967376 1556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:22.970765 1556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:22.970973 1556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:22.978281 1556 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41921
I20260619 18:51:22.978325 1623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41921 every 8 connection(s)
I20260619 18:51:22.979272 1556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:22.982110 1624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:22.985509 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 1556
I20260619 18:51:22.985663 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:22.985946 13227 external_mini_cluster.cc:1428] Setting key 227cb1030b4157fbd95446aebeb89441
I20260619 18:51:22.988334 1624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390: Bootstrap starting.
I20260619 18:51:22.990978 1624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:22.991679 1627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:22.987452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50829 (local address 127.0.0.1:41921)
0619 18:51:22.987889 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:22.987906 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:22.987953 (+ 47us) server_negotiation.cc:407] Connection header received
0619 18:51:22.988656 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:22.988676 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:22.989010 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:22.989325 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:22.990291 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:22.990296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:22.990309 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:22.990322 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:22.990382 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:22.990404 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:22.990507 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:22.990685 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 18:51:22.990861 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":114,"threads_started":1}
I20260619 18:51:22.991725 1624 log.cc:824] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:22.993790 1624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390: No bootstrap required, opened a new log
I20260619 18:51:22.994441 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:22.998585 1627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:22.996818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55065 (local address 127.0.0.1:41921)
0619 18:51:22.996945 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:22.996951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:22.997259 (+ 308us) server_negotiation.cc:407] Connection header received
0619 18:51:22.997390 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:22.997392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:22.997426 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:22.997492 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:22.998215 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:22.998219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:22.998222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:22.998235 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:22.998261 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:22.998273 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:22.998343 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:22.998430 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 18:51:22.998463 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 18:51:22.998631 1624 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec155618bd124af090642ea62b43d390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } }
I20260619 18:51:22.998951 1624 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:22.999020 1624 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec155618bd124af090642ea62b43d390, State: Initialized, Role: FOLLOWER
I20260619 18:51:22.999513 1624 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec155618bd124af090642ea62b43d390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } }
I20260619 18:51:22.999652 1624 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:22.999718 1624 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:22.999815 1624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:23.000787 1624 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec155618bd124af090642ea62b43d390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } }
I20260619 18:51:23.001223 1624 leader_election.cc:302] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec155618bd124af090642ea62b43d390; no voters:
I20260619 18:51:23.001576 1624 leader_election.cc:288] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:23.001648 1634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:23.001842 1634 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [term 1 LEADER]: Becoming Leader. State: Replica: ec155618bd124af090642ea62b43d390, State: Running, Role: LEADER
I20260619 18:51:23.002208 1634 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec155618bd124af090642ea62b43d390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } }
I20260619 18:51:23.002918 1624 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:23.003891 1635 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec155618bd124af090642ea62b43d390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec155618bd124af090642ea62b43d390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } } }
I20260619 18:51:23.003891 1636 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec155618bd124af090642ea62b43d390. Latest consensus state: current_term: 1 leader_uuid: "ec155618bd124af090642ea62b43d390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec155618bd124af090642ea62b43d390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } } }
I20260619 18:51:23.004042 1636 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:23.004042 1635 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:23.004490 1643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:23.007486 1643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:23.012945 1643 catalog_manager.cc:1347] Generated new cluster ID: 1c52fbb7f34b4846bcbe8467e9fd6caf
I20260619 18:51:23.013032 1643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:23.030437 1643 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:23.030870 1643 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:23.038123 1643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec155618bd124af090642ea62b43d390: Generated new TSK 0
I20260619 18:51:23.039108 1643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:23.044891 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 1556
2026-06-19T18:51:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:23 dist-test-slave-fljv krb5kdc[1655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:23 dist-test-slave-fljv krb5kdc[1655](info): set up 2 sockets
Jun 19 18:51:23 dist-test-slave-fljv krb5kdc[1655](info): commencing operation
krb5kdc: starting...
W20260619 18:51:24.010416 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:24 dist-test-slave-fljv krb5kdc[1655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:24.133210 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:24.241763 1668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:24.242041 1668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:24.242084 1668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:24.246057 1668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:24.246129 1668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:24.246148 1668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:24.246163 1668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:24.246197 1668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:24.251554 1668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:24.252619 1668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:24.253640 1668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:24.262444 1674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:24.262542 1668 server_base.cc:1034] running on GCE node
W20260619 18:51:24.262444 1676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:24.262455 1673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:24.263335 1668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:24.264461 1668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:24.265617 1668 hybrid_clock.cc:648] HybridClock initialized: now 1781895084265589 us; error 48 us; skew 500 ppm
Jun 19 18:51:24 dist-test-slave-fljv krb5kdc[1655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895084, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:24.268340 1668 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:24.269308 1668 webserver.cc:460] Webserver started at http://127.0.0.1:40887/ using document root <none> and password file <none>
I20260619 18:51:24.269855 1668 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:24.269920 1668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:24.270115 1668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:24.272198 1668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "144e59020765481d8677b367f16905f4"
format_stamp: "Formatted at 2026-06-19 18:51:24 on dist-test-slave-fljv"
server_key: "52f0af0627cff1ff890a105f160cc12b"
server_key_iv: "c1f2bf1f8139c5dc77d2f52db72a3fba"
server_key_version: "encryptionkey@0"
I20260619 18:51:24.272753 1668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "144e59020765481d8677b367f16905f4"
format_stamp: "Formatted at 2026-06-19 18:51:24 on dist-test-slave-fljv"
server_key: "52f0af0627cff1ff890a105f160cc12b"
server_key_iv: "c1f2bf1f8139c5dc77d2f52db72a3fba"
server_key_version: "encryptionkey@0"
I20260619 18:51:24.276542 1668 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260619 18:51:24.278980 1683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:24.280216 1668 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 18:51:24.280331 1668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "144e59020765481d8677b367f16905f4"
format_stamp: "Formatted at 2026-06-19 18:51:24 on dist-test-slave-fljv"
server_key: "52f0af0627cff1ff890a105f160cc12b"
server_key_iv: "c1f2bf1f8139c5dc77d2f52db72a3fba"
server_key_version: "encryptionkey@0"
I20260619 18:51:24.280428 1668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:24.298633 1668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:24.301597 1668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:24.301782 1668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:24.308650 1668 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37079
I20260619 18:51:24.308707 1735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37079 every 8 connection(s)
I20260619 18:51:24.309669 1668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:24.312456 1736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:24.318213 1736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4: Bootstrap starting.
I20260619 18:51:24.319427 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 1668
I20260619 18:51:24.319578 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:24.319864 13227 external_mini_cluster.cc:1428] Setting key 78da852c0de5dbd5a3203a753c26eb01
I20260619 18:51:24.321132 1736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:24.321877 1736 log.cc:824] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:24.324015 1736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4: No bootstrap required, opened a new log
I20260619 18:51:24.328219 1736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144e59020765481d8677b367f16905f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } }
I20260619 18:51:24.328517 1736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:24.328583 1736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144e59020765481d8677b367f16905f4, State: Initialized, Role: FOLLOWER
Jun 19 18:51:24 dist-test-slave-fljv krb5kdc[1655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 18:51:24.329051 1736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144e59020765481d8677b367f16905f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } }
I20260619 18:51:24.329195 1736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:24.329259 1736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:24.329357 1736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:24.330314 1736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144e59020765481d8677b367f16905f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } }
I20260619 18:51:24.330714 1736 leader_election.cc:302] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 144e59020765481d8677b367f16905f4; no voters:
I20260619 18:51:24.331055 1736 leader_election.cc:288] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:24.331156 1741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:24.331318 1741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [term 1 LEADER]: Becoming Leader. State: Replica: 144e59020765481d8677b367f16905f4, State: Running, Role: LEADER
I20260619 18:51:24.331688 1741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144e59020765481d8677b367f16905f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } }
I20260619 18:51:24.332461 1736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:24.333577 1743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "144e59020765481d8677b367f16905f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144e59020765481d8677b367f16905f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } } }
I20260619 18:51:24.333985 1743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:24.333876 1742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 144e59020765481d8677b367f16905f4. Latest consensus state: current_term: 1 leader_uuid: "144e59020765481d8677b367f16905f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144e59020765481d8677b367f16905f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } } }
I20260619 18:51:24.334981 1742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:24.335961 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:24.321050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:50991 (local address 127.0.0.1:37079)
0619 18:51:24.321476 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:24.321492 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:24.321530 (+ 38us) server_negotiation.cc:407] Connection header received
0619 18:51:24.322257 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:24.322277 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:24.322631 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:24.322916 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:24.323776 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:24.324842 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:24.326053 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:24.326567 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:24.326686 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:24.329691 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:24.329696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:24.329710 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:24.329740 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:24.331749 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:24.332422 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:24.332429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:24.332436 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:24.332644 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:24.333031 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:24.333035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:24.333037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:24.333623 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:24.333905 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:24.334823 (+ 918us) server_negotiation.cc:299] Negotiation successful
0619 18:51:24.335079 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
W20260619 18:51:24.337973 1757 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:51:24.338080 1757 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:24.338590 1754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:24.339493 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:24.339552 1754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:24.346376 1754 catalog_manager.cc:1347] Generated new cluster ID: 058db19d1f37466890a24813ec167970
I20260619 18:51:24.346473 1754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:24.355141 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:24.341558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:41035 (local address 127.0.0.1:37079)
0619 18:51:24.341680 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:24.341685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:24.342069 (+ 384us) server_negotiation.cc:407] Connection header received
0619 18:51:24.342149 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:24.342152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:24.342242 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:24.342327 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:24.343284 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:24.344115 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:24.345260 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:24.345632 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:24.345743 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:24.351096 (+ 5353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:24.351102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:24.351105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:24.351144 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:24.352849 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:24.353507 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:24.353511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:24.353513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:24.353609 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:24.353932 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:24.353935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:24.353937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:24.354208 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:24.354360 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:24.354627 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 18:51:24.354740 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 18:51:24.371971 1754 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:24.373433 1754 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:24.385234 1754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 144e59020765481d8677b367f16905f4: Generated new TSK 0
I20260619 18:51:24.386005 1754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:24.393842 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 1668
W20260619 18:51:24.398614 1653 connection.cc:535] client connection to 127.0.0.1:37079 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37079): Connection reset by peer (error 104)
2026-06-19T18:51:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:24 dist-test-slave-fljv krb5kdc[1767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:24 dist-test-slave-fljv krb5kdc[1767](info): set up 2 sockets
Jun 19 18:51:24 dist-test-slave-fljv krb5kdc[1767](info): commencing operation
krb5kdc: starting...
W20260619 18:51:25.362221 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:25 dist-test-slave-fljv krb5kdc[1767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:25.479557 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37483
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:25.593055 1780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:25.593343 1780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:25.593385 1780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:25.597513 1780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:25.597599 1780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:25.597617 1780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:25.597633 1780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:25.597647 1780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:25.603092 1780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:25.604140 1780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:25.605178 1780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:25.613649 1786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:25.613761 1780 server_base.cc:1034] running on GCE node
W20260619 18:51:25.613634 1785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:25.613685 1788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:25.614673 1780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:25.615816 1780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:25.616981 1780 hybrid_clock.cc:648] HybridClock initialized: now 1781895085616963 us; error 32 us; skew 500 ppm
Jun 19 18:51:25 dist-test-slave-fljv krb5kdc[1767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895085, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:25.627576 1780 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:25.628708 1780 webserver.cc:460] Webserver started at http://127.0.0.1:35927/ using document root <none> and password file <none>
I20260619 18:51:25.629312 1780 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:25.629395 1780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:25.629617 1780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:25.631822 1780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "ed3998962d0045acbb72b9919b017fcc"
format_stamp: "Formatted at 2026-06-19 18:51:25 on dist-test-slave-fljv"
server_key: "c2c949746dd59e983709cd78a26a3108"
server_key_iv: "eea39deb6a1075f360d12ae379bcd1d2"
server_key_version: "encryptionkey@0"
I20260619 18:51:25.632370 1780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "ed3998962d0045acbb72b9919b017fcc"
format_stamp: "Formatted at 2026-06-19 18:51:25 on dist-test-slave-fljv"
server_key: "c2c949746dd59e983709cd78a26a3108"
server_key_iv: "eea39deb6a1075f360d12ae379bcd1d2"
server_key_version: "encryptionkey@0"
I20260619 18:51:25.636222 1780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:51:25.638829 1795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:25.640187 1780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:25.640314 1780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "ed3998962d0045acbb72b9919b017fcc"
format_stamp: "Formatted at 2026-06-19 18:51:25 on dist-test-slave-fljv"
server_key: "c2c949746dd59e983709cd78a26a3108"
server_key_iv: "eea39deb6a1075f360d12ae379bcd1d2"
server_key_version: "encryptionkey@0"
I20260619 18:51:25.640424 1780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:25.671314 1780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:25.674364 1780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:25.674554 1780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:25.681605 1780 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37483
I20260619 18:51:25.681619 1847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37483 every 8 connection(s)
I20260619 18:51:25.682626 1780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:25.685433 1848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:25.686015 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 1780
I20260619 18:51:25.686141 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:25.686436 13227 external_mini_cluster.cc:1428] Setting key e8e3635e47ffb4b21d23e75288401b22
I20260619 18:51:25.691782 1848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc: Bootstrap starting.
I20260619 18:51:25.694072 1848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:25.694850 1848 log.cc:824] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:25.695796 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:25.687824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:52835 (local address 127.0.0.1:37483)
0619 18:51:25.688309 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:25.688328 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:25.688525 (+ 197us) server_negotiation.cc:407] Connection header received
0619 18:51:25.689288 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:25.689306 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:25.689696 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:25.690019 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:25.690464 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:25.691436 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:25.692379 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:25.692871 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:25.693019 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:25.693876 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:25.693881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:25.693895 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:25.694354 (+ 459us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:25.694422 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:25.694446 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:25.694566 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:25.694752 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 18:51:25.694894 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":158,"threads_started":1}
I20260619 18:51:25.697224 1848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc: No bootstrap required, opened a new log
I20260619 18:51:25.698990 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:25.701212 1848 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed3998962d0045acbb72b9919b017fcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } }
I20260619 18:51:25.701508 1848 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:25.701617 1848 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed3998962d0045acbb72b9919b017fcc, State: Initialized, Role: FOLLOWER
I20260619 18:51:25.702122 1848 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed3998962d0045acbb72b9919b017fcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } }
I20260619 18:51:25.702303 1848 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:25.702365 1848 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:25.702457 1848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:25.703444 1848 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed3998962d0045acbb72b9919b017fcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } }
I20260619 18:51:25.703825 1848 leader_election.cc:302] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed3998962d0045acbb72b9919b017fcc; no voters:
I20260619 18:51:25.704192 1848 leader_election.cc:288] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:25.704260 1858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:25.704474 1858 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [term 1 LEADER]: Becoming Leader. State: Replica: ed3998962d0045acbb72b9919b017fcc, State: Running, Role: LEADER
I20260619 18:51:25.704842 1858 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed3998962d0045acbb72b9919b017fcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } }
I20260619 18:51:25.705268 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:25.700937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:35511 (local address 127.0.0.1:37483)
0619 18:51:25.701060 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:25.701065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:25.701270 (+ 205us) server_negotiation.cc:407] Connection header received
0619 18:51:25.701360 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:25.701363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:25.701413 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:25.701511 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:25.701805 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:25.702585 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:25.703415 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:25.703770 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:25.703874 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:25.704795 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:25.704799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:25.704802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:25.704816 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:25.704846 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:25.704860 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:25.704935 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:25.705103 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 18:51:25.705137 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:51:25.705564 1848 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:25.706707 1860 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed3998962d0045acbb72b9919b017fcc. Latest consensus state: current_term: 1 leader_uuid: "ed3998962d0045acbb72b9919b017fcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed3998962d0045acbb72b9919b017fcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } } }
I20260619 18:51:25.706702 1859 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed3998962d0045acbb72b9919b017fcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed3998962d0045acbb72b9919b017fcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } } }
I20260619 18:51:25.706841 1860 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:25.706846 1859 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:25.707211 1867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:25.710413 1867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:25.717082 1867 catalog_manager.cc:1347] Generated new cluster ID: 901f2413532d44da8c9986e94bd614c1
I20260619 18:51:25.717156 1867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:25.737222 1867 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:25.738242 1867 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:25.751971 1867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed3998962d0045acbb72b9919b017fcc: Generated new TSK 0
I20260619 18:51:25.752754 1867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:25.762669 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 1780
W20260619 18:51:25.767520 1765 connection.cc:535] client connection to 127.0.0.1:37483 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37483): Connection reset by peer (error 104)
2026-06-19T18:51:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:25 dist-test-slave-fljv krb5kdc[1879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:25 dist-test-slave-fljv krb5kdc[1879](info): set up 2 sockets
Jun 19 18:51:25 dist-test-slave-fljv krb5kdc[1879](info): commencing operation
krb5kdc: starting...
W20260619 18:51:26.718288 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:26 dist-test-slave-fljv krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:26.838286 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:26.949893 1892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:26.950228 1892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:26.950299 1892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:26.954571 1892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:26.954648 1892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:26.954676 1892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:26.954703 1892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:26.954733 1892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:26.960203 1892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:26.961349 1892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:26.962471 1892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:26.971246 1897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:26.971220 1900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:26.971380 1892 server_base.cc:1034] running on GCE node
W20260619 18:51:26.971222 1898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:26.972115 1892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:26.973253 1892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:26.974447 1892 hybrid_clock.cc:648] HybridClock initialized: now 1781895086974429 us; error 32 us; skew 500 ppm
Jun 19 18:51:26 dist-test-slave-fljv krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895086, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:26.977334 1892 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:26.978317 1892 webserver.cc:460] Webserver started at http://127.0.0.1:43005/ using document root <none> and password file <none>
I20260619 18:51:26.978863 1892 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:26.978932 1892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:26.979146 1892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:26.981241 1892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "a38a58e417ab4601be22f3b5888c4ec2"
format_stamp: "Formatted at 2026-06-19 18:51:26 on dist-test-slave-fljv"
server_key: "f609b3891adc832cffa7f49140fcd779"
server_key_iv: "4cc5400d4ed2b8cda58829fd6ec9ce11"
server_key_version: "encryptionkey@0"
I20260619 18:51:26.981779 1892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "a38a58e417ab4601be22f3b5888c4ec2"
format_stamp: "Formatted at 2026-06-19 18:51:26 on dist-test-slave-fljv"
server_key: "f609b3891adc832cffa7f49140fcd779"
server_key_iv: "4cc5400d4ed2b8cda58829fd6ec9ce11"
server_key_version: "encryptionkey@0"
I20260619 18:51:26.985548 1892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 18:51:26.988181 1907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:26.989511 1892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:51:26.989645 1892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "a38a58e417ab4601be22f3b5888c4ec2"
format_stamp: "Formatted at 2026-06-19 18:51:26 on dist-test-slave-fljv"
server_key: "f609b3891adc832cffa7f49140fcd779"
server_key_iv: "4cc5400d4ed2b8cda58829fd6ec9ce11"
server_key_version: "encryptionkey@0"
I20260619 18:51:26.989758 1892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:26.994611 1892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:26.997730 1892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:26.997901 1892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:27.004735 1892 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40123
I20260619 18:51:27.004745 1959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40123 every 8 connection(s)
I20260619 18:51:27.005673 1892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:27.008562 1960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:27.014264 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 1892
I20260619 18:51:27.014420 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:27.014415 1960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2: Bootstrap starting.
I20260619 18:51:27.014750 13227 external_mini_cluster.cc:1428] Setting key dc2399a330f6a906d58ddebb6ad6fd53
I20260619 18:51:27.017081 1960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:27.017830 1960 log.cc:824] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:27.020040 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:27.016182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:38437 (local address 127.0.0.1:40123)
0619 18:51:27.016620 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:27.016635 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:27.016674 (+ 39us) server_negotiation.cc:407] Connection header received
0619 18:51:27.017373 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:27.017392 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:27.017700 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:27.018028 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:27.018782 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:27.018787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:27.018799 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:27.018809 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:27.018862 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:27.018881 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:27.018968 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:27.019139 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 18:51:27.019320 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":116,"threads_started":1}
I20260619 18:51:27.020044 1960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2: No bootstrap required, opened a new log
I20260619 18:51:27.022873 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:27.024065 1960 raft_consensus.cc:348] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38a58e417ab4601be22f3b5888c4ec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260619 18:51:27.024406 1960 raft_consensus.cc:374] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:27.024482 1960 raft_consensus.cc:729] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38a58e417ab4601be22f3b5888c4ec2, State: Initialized, Role: FOLLOWER
I20260619 18:51:27.024986 1960 consensus_queue.cc:260] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38a58e417ab4601be22f3b5888c4ec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260619 18:51:27.025151 1960 raft_consensus.cc:388] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:27.025251 1960 raft_consensus.cc:482] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:27.025343 1960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:27.026414 1960 raft_consensus.cc:504] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38a58e417ab4601be22f3b5888c4ec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260619 18:51:27.026808 1960 leader_election.cc:302] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a38a58e417ab4601be22f3b5888c4ec2; no voters:
I20260619 18:51:27.027043 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:27.025187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:59269 (local address 127.0.0.1:40123)
0619 18:51:27.025296 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:27.025301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:27.025682 (+ 381us) server_negotiation.cc:407] Connection header received
0619 18:51:27.025800 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:27.025803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:27.025841 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:27.025907 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:27.026655 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:27.026659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:27.026662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:27.026675 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:27.026706 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:27.026718 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:27.026794 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:27.026877 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 18:51:27.026920 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 18:51:27.027143 1960 leader_election.cc:288] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:27.027221 1970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:27.027406 1970 raft_consensus.cc:686] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [term 1 LEADER]: Becoming Leader. State: Replica: a38a58e417ab4601be22f3b5888c4ec2, State: Running, Role: LEADER
I20260619 18:51:27.027761 1970 consensus_queue.cc:237] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38a58e417ab4601be22f3b5888c4ec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260619 18:51:27.028554 1960 sys_catalog.cc:564] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:27.029583 1972 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a38a58e417ab4601be22f3b5888c4ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38a58e417ab4601be22f3b5888c4ec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } } }
I20260619 18:51:27.029727 1972 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:27.030030 1971 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a38a58e417ab4601be22f3b5888c4ec2. Latest consensus state: current_term: 1 leader_uuid: "a38a58e417ab4601be22f3b5888c4ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38a58e417ab4601be22f3b5888c4ec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } } }
I20260619 18:51:27.030109 1979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:27.033432 1979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:27.035694 1971 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:27.039324 1979 catalog_manager.cc:1347] Generated new cluster ID: 2bd26915b6214ba9907c199cc2458ece
I20260619 18:51:27.039552 1979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:27.052987 1979 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:27.053335 1979 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:27.057729 1979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a38a58e417ab4601be22f3b5888c4ec2: Generated new TSK 0
I20260619 18:51:27.058482 1979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:27.066478 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 1892
2026-06-19T18:51:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:27 dist-test-slave-fljv krb5kdc[1991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:27 dist-test-slave-fljv krb5kdc[1991](info): set up 2 sockets
Jun 19 18:51:27 dist-test-slave-fljv krb5kdc[1991](info): commencing operation
krb5kdc: starting...
W20260619 18:51:28.018458 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:28 dist-test-slave-fljv krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:28.138562 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:28.250586 2004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:28.250916 2004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:28.250983 2004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:28.255394 2004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:28.255475 2004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:28.255502 2004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:28.255524 2004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:28.255553 2004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:28.261073 2004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43859
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:28.262295 2004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:28.263350 2004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:28.272362 2009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:28.272362 2012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:28.272442 2004 server_base.cc:1034] running on GCE node
W20260619 18:51:28.272362 2010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:28.273293 2004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:28.274549 2004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:28.275731 2004 hybrid_clock.cc:648] HybridClock initialized: now 1781895088275706 us; error 40 us; skew 500 ppm
Jun 19 18:51:28 dist-test-slave-fljv krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895088, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:28.278556 2004 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:28.279484 2004 webserver.cc:460] Webserver started at http://127.0.0.1:45561/ using document root <none> and password file <none>
I20260619 18:51:28.280040 2004 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:28.280107 2004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:28.280321 2004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:28.282465 2004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "638df75c966644c58fc6a0ab6c6171f2"
format_stamp: "Formatted at 2026-06-19 18:51:28 on dist-test-slave-fljv"
server_key: "a4055d8a298d70f4fe947c5a68cd4b66"
server_key_iv: "fae6756330d5ba1a3792157c37396256"
server_key_version: "encryptionkey@0"
I20260619 18:51:28.283021 2004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "638df75c966644c58fc6a0ab6c6171f2"
format_stamp: "Formatted at 2026-06-19 18:51:28 on dist-test-slave-fljv"
server_key: "a4055d8a298d70f4fe947c5a68cd4b66"
server_key_iv: "fae6756330d5ba1a3792157c37396256"
server_key_version: "encryptionkey@0"
I20260619 18:51:28.286918 2004 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:51:28.289431 2019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:28.290807 2004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:51:28.290912 2004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "638df75c966644c58fc6a0ab6c6171f2"
format_stamp: "Formatted at 2026-06-19 18:51:28 on dist-test-slave-fljv"
server_key: "a4055d8a298d70f4fe947c5a68cd4b66"
server_key_iv: "fae6756330d5ba1a3792157c37396256"
server_key_version: "encryptionkey@0"
I20260619 18:51:28.291015 2004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:28.307664 2004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:28.310953 2004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:28.311141 2004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:28.318226 2004 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43859
I20260619 18:51:28.318277 2071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43859 every 8 connection(s)
I20260619 18:51:28.319273 2004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:28.322149 2072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:28.325027 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 2004
I20260619 18:51:28.325169 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:28.325495 13227 external_mini_cluster.cc:1428] Setting key 8e2f77a003a75aded4be567042e7614c
I20260619 18:51:28.328402 2072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2: Bootstrap starting.
I20260619 18:51:28.331266 2072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:28.332186 2072 log.cc:824] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:28.334262 2072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2: No bootstrap required, opened a new log
Jun 19 18:51:28 dist-test-slave-fljv krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 18:51:28.338281 2072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638df75c966644c58fc6a0ab6c6171f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260619 18:51:28.338589 2072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:28.338689 2072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638df75c966644c58fc6a0ab6c6171f2, State: Initialized, Role: FOLLOWER
I20260619 18:51:28.339238 2072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638df75c966644c58fc6a0ab6c6171f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260619 18:51:28.339411 2072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:28.339479 2072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:28.339579 2072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:28.340833 2072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638df75c966644c58fc6a0ab6c6171f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260619 18:51:28.341228 2072 leader_election.cc:302] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 638df75c966644c58fc6a0ab6c6171f2; no voters:
I20260619 18:51:28.341578 2072 leader_election.cc:288] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:28.341714 2077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:28.341917 2077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [term 1 LEADER]: Becoming Leader. State: Replica: 638df75c966644c58fc6a0ab6c6171f2, State: Running, Role: LEADER
I20260619 18:51:28.342087 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:28.326925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:58543 (local address 127.0.0.1:43859)
0619 18:51:28.327409 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:28.327424 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:28.327460 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:51:28.328139 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:28.328157 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:28.328535 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:28.328853 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:28.329757 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:28.330909 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:28.332112 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:28.332656 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:28.332825 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:28.335813 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:28.335820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:28.335840 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:28.335876 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:28.338514 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:28.339082 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:28.339088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:28.339094 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:28.339245 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:28.339664 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:28.339667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:28.339669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:28.340197 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:28.340416 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:28.340753 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 18:51:28.341000 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":133,"threads_started":1}
I20260619 18:51:28.342401 2077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638df75c966644c58fc6a0ab6c6171f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260619 18:51:28.342917 2072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:28.343627 2077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 638df75c966644c58fc6a0ab6c6171f2. Latest consensus state: current_term: 1 leader_uuid: "638df75c966644c58fc6a0ab6c6171f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638df75c966644c58fc6a0ab6c6171f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } } }
I20260619 18:51:28.343763 2077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:28.344035 2079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "638df75c966644c58fc6a0ab6c6171f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638df75c966644c58fc6a0ab6c6171f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } } }
I20260619 18:51:28.344141 2079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:28.344257 2085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:28.345660 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:28.347731 2085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:28.353381 2085 catalog_manager.cc:1347] Generated new cluster ID: d87ed248a12a4f8c8f449a6c0ade3306
I20260619 18:51:28.353457 2085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:28.358251 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:28.348071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:35789 (local address 127.0.0.1:43859)
0619 18:51:28.348196 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:28.348201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:28.348774 (+ 573us) server_negotiation.cc:407] Connection header received
0619 18:51:28.348971 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:28.348974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:28.349027 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:28.349110 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:28.349965 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:28.350807 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:28.352029 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:28.352401 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:28.352477 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:28.355083 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:28.355086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:28.355089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:28.355114 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:28.356524 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:28.356994 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:28.356997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:28.356998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:28.357081 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:28.357384 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:28.357387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:28.357388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:28.357562 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:28.357705 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:28.357951 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 18:51:28.358054 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 18:51:28.375979 2085 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:28.376940 2085 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:28.381583 2085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 638df75c966644c58fc6a0ab6c6171f2: Generated new TSK 0
I20260619 18:51:28.382351 2085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:28.386593 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 2004
W20260619 18:51:28.391489 1989 connection.cc:535] client connection to 127.0.0.1:43859 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43859): Connection reset by peer (error 104)
2026-06-19T18:51:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:28 dist-test-slave-fljv krb5kdc[2103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:28 dist-test-slave-fljv krb5kdc[2103](info): set up 2 sockets
Jun 19 18:51:28 dist-test-slave-fljv krb5kdc[2103](info): commencing operation
krb5kdc: starting...
W20260619 18:51:29.342308 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:29 dist-test-slave-fljv krb5kdc[2103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:29.461511 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:29.576870 2116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:29.577165 2116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:29.577212 2116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:29.581380 2116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:29.581465 2116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:29.581485 2116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:29.581501 2116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:29.581534 2116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:29.587062 2116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32923
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:29.588231 2116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:29.589344 2116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:29.598145 2121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:29.598253 2122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:29.598248 2116 server_base.cc:1034] running on GCE node
W20260619 18:51:29.598408 2124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:29.598968 2116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:29.600092 2116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:29.601281 2116 hybrid_clock.cc:648] HybridClock initialized: now 1781895089601262 us; error 36 us; skew 500 ppm
Jun 19 18:51:29 dist-test-slave-fljv krb5kdc[2103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895089, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:29.604238 2116 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:29.605289 2116 webserver.cc:460] Webserver started at http://127.0.0.1:40029/ using document root <none> and password file <none>
I20260619 18:51:29.605876 2116 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:29.605952 2116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:29.606225 2116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:29.608453 2116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "e57cd76ff7374c06bae52ce5d6038052"
format_stamp: "Formatted at 2026-06-19 18:51:29 on dist-test-slave-fljv"
server_key: "92b221a6612f9c8080b532da217ff82c"
server_key_iv: "5ad5ec4e437ced98f2ff78e6c35984ad"
server_key_version: "encryptionkey@0"
I20260619 18:51:29.609081 2116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "e57cd76ff7374c06bae52ce5d6038052"
format_stamp: "Formatted at 2026-06-19 18:51:29 on dist-test-slave-fljv"
server_key: "92b221a6612f9c8080b532da217ff82c"
server_key_iv: "5ad5ec4e437ced98f2ff78e6c35984ad"
server_key_version: "encryptionkey@0"
I20260619 18:51:29.613240 2116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 18:51:29.615981 2131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:29.617511 2116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:51:29.617647 2116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "e57cd76ff7374c06bae52ce5d6038052"
format_stamp: "Formatted at 2026-06-19 18:51:29 on dist-test-slave-fljv"
server_key: "92b221a6612f9c8080b532da217ff82c"
server_key_iv: "5ad5ec4e437ced98f2ff78e6c35984ad"
server_key_version: "encryptionkey@0"
I20260619 18:51:29.617766 2116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:29.632598 2116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:29.635900 2116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:29.636097 2116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:29.643587 2116 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32923
I20260619 18:51:29.643595 2183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32923 every 8 connection(s)
I20260619 18:51:29.644584 2116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:29.647461 2184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:29.648159 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 2116
I20260619 18:51:29.648293 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:29.648603 13227 external_mini_cluster.cc:1428] Setting key b8980b8c4b05b6aaaa9f18f00b55d206
I20260619 18:51:29.653638 2184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052: Bootstrap starting.
I20260619 18:51:29.656126 2184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:29.656843 2184 log.cc:824] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:29.657857 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:29.650104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:47399 (local address 127.0.0.1:32923)
0619 18:51:29.650579 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:29.650595 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:29.650632 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:51:29.651297 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:29.651313 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:29.651670 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:29.652006 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:29.652443 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:29.653414 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:29.654473 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:29.654942 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:29.655134 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:29.656014 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:29.656019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:29.656032 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:29.656477 (+ 445us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:29.656533 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:29.656556 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:29.656669 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:29.656861 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 18:51:29.657007 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":136,"threads_started":1}
I20260619 18:51:29.658982 2184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052: No bootstrap required, opened a new log
I20260619 18:51:29.660861 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:29.662756 2184 raft_consensus.cc:348] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57cd76ff7374c06bae52ce5d6038052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32923 } }
I20260619 18:51:29.663091 2184 raft_consensus.cc:374] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:29.663158 2184 raft_consensus.cc:729] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e57cd76ff7374c06bae52ce5d6038052, State: Initialized, Role: FOLLOWER
I20260619 18:51:29.663614 2184 consensus_queue.cc:260] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57cd76ff7374c06bae52ce5d6038052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32923 } }
I20260619 18:51:29.663745 2184 raft_consensus.cc:388] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:29.663807 2184 raft_consensus.cc:482] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:29.663895 2184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:29.664839 2184 raft_consensus.cc:504] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57cd76ff7374c06bae52ce5d6038052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32923 } }
I20260619 18:51:29.665177 2184 leader_election.cc:302] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e57cd76ff7374c06bae52ce5d6038052; no voters:
I20260619 18:51:29.665477 2184 leader_election.cc:288] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:29.665565 2194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:29.665766 2194 raft_consensus.cc:686] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [term 1 LEADER]: Becoming Leader. State: Replica: e57cd76ff7374c06bae52ce5d6038052, State: Running, Role: LEADER
I20260619 18:51:29.666095 2194 consensus_queue.cc:237] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57cd76ff7374c06bae52ce5d6038052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32923 } }
I20260619 18:51:29.666965 2184 sys_catalog.cc:564] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:29.667007 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:29.662814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:60829 (local address 127.0.0.1:32923)
0619 18:51:29.662943 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:29.662948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:29.663219 (+ 271us) server_negotiation.cc:407] Connection header received
0619 18:51:29.663279 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:29.663282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:29.663328 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:29.663404 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:29.663711 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:29.664383 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:29.665289 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:29.665602 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:29.665694 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:29.666568 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:29.666572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:29.666574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:29.666588 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:29.666614 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:29.666625 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:29.666707 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:29.666844 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 18:51:29.666878 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 18:51:29.667398 2194 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e57cd76ff7374c06bae52ce5d6038052. Latest consensus state: current_term: 1 leader_uuid: "e57cd76ff7374c06bae52ce5d6038052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57cd76ff7374c06bae52ce5d6038052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32923 } } }
I20260619 18:51:29.667520 2194 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:29.667964 2196 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e57cd76ff7374c06bae52ce5d6038052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57cd76ff7374c06bae52ce5d6038052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32923 } } }
I20260619 18:51:29.668119 2196 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:29.668440 2198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:29.672051 2198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:29.678892 2198 catalog_manager.cc:1347] Generated new cluster ID: db838f9ddb6d46a8ac34da96c6937e87
I20260619 18:51:29.679075 2198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:29.709896 2198 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:29.711344 2198 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:29.719118 2198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e57cd76ff7374c06bae52ce5d6038052: Generated new TSK 0
I20260619 18:51:29.720024 2198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:29.726670 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 2116
W20260619 18:51:29.731225 2101 connection.cc:535] client connection to 127.0.0.1:32923 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32923): Connection reset by peer (error 104)
2026-06-19T18:51:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:29 dist-test-slave-fljv krb5kdc[2215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:29 dist-test-slave-fljv krb5kdc[2215](info): set up 2 sockets
Jun 19 18:51:29 dist-test-slave-fljv krb5kdc[2215](info): commencing operation
krb5kdc: starting...
W20260619 18:51:30.678282 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:30 dist-test-slave-fljv krb5kdc[2215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 18:51:30.801065 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45093
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:30.914239 2228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:30.914554 2228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:30.914614 2228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:30.919078 2228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:30.919175 2228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:30.919203 2228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:30.919236 2228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:30.919265 2228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:30.925040 2228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45093
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:30.926265 2228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:30.927317 2228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:30.936070 2236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:30.936065 2233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:30.936244 2228 server_base.cc:1034] running on GCE node
W20260619 18:51:30.936136 2234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:30.936863 2228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:30.938055 2228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:30.939249 2228 hybrid_clock.cc:648] HybridClock initialized: now 1781895090939222 us; error 39 us; skew 500 ppm
Jun 19 18:51:30 dist-test-slave-fljv krb5kdc[2215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895090, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:30.946786 2228 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 18:51:30.947938 2228 webserver.cc:460] Webserver started at http://127.0.0.1:41569/ using document root <none> and password file <none>
I20260619 18:51:30.948544 2228 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:30.948616 2228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:30.948849 2228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:30.951156 2228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "fe8e85700d934e4cae88a26384ee5129"
format_stamp: "Formatted at 2026-06-19 18:51:30 on dist-test-slave-fljv"
server_key: "261d9178684dd0059959c38c13854895"
server_key_iv: "0d2814926ae1148716cea660a98979c5"
server_key_version: "encryptionkey@0"
I20260619 18:51:30.951727 2228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "fe8e85700d934e4cae88a26384ee5129"
format_stamp: "Formatted at 2026-06-19 18:51:30 on dist-test-slave-fljv"
server_key: "261d9178684dd0059959c38c13854895"
server_key_iv: "0d2814926ae1148716cea660a98979c5"
server_key_version: "encryptionkey@0"
I20260619 18:51:30.955860 2228 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:30.958488 2243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:30.959826 2228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 18:51:30.959949 2228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "fe8e85700d934e4cae88a26384ee5129"
format_stamp: "Formatted at 2026-06-19 18:51:30 on dist-test-slave-fljv"
server_key: "261d9178684dd0059959c38c13854895"
server_key_iv: "0d2814926ae1148716cea660a98979c5"
server_key_version: "encryptionkey@0"
I20260619 18:51:30.960052 2228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:30.964768 2228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:30.967990 2228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:30.968178 2228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:30.975193 2228 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45093
I20260619 18:51:30.975229 2295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45093 every 8 connection(s)
I20260619 18:51:30.976158 2228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:30.977420 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 2228
I20260619 18:51:30.977533 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:30.977828 13227 external_mini_cluster.cc:1428] Setting key 0c37bb524267fa2fb373e9a639af62bf
I20260619 18:51:30.979228 2296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:30.982950 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:30.979140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:56605 (local address 127.0.0.1:45093)
0619 18:51:30.979544 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:30.979560 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:30.979597 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:51:30.980240 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:30.980256 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:30.980545 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:30.980830 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:30.981683 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:30.981688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:30.981700 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:30.981710 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:30.981764 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:30.981784 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:30.981888 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:30.981983 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 18:51:30.982133 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":126,"threads_started":1}
I20260619 18:51:30.985388 2296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129: Bootstrap starting.
I20260619 18:51:30.986025 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:30.988237 2296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:30.989164 2296 log.cc:824] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:30.990309 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:30.988437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.71:46011 (local address 127.0.0.1:45093)
0619 18:51:30.988580 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:30.988584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:30.988836 (+ 252us) server_negotiation.cc:407] Connection header received
0619 18:51:30.988946 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:30.988949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:30.988992 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:30.989093 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:30.989855 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:30.989858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:30.989860 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:30.989871 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:30.989896 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:30.989906 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:30.989973 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:30.990122 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 18:51:30.990166 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 18:51:30.991559 2296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129: No bootstrap required, opened a new log
I20260619 18:51:30.995096 2296 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8e85700d934e4cae88a26384ee5129" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } }
I20260619 18:51:30.995344 2296 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:30.995383 2296 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe8e85700d934e4cae88a26384ee5129, State: Initialized, Role: FOLLOWER
I20260619 18:51:30.995821 2296 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8e85700d934e4cae88a26384ee5129" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } }
I20260619 18:51:30.995963 2296 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:30.996028 2296 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:30.996129 2296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:30.997191 2296 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8e85700d934e4cae88a26384ee5129" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } }
I20260619 18:51:30.997565 2296 leader_election.cc:302] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe8e85700d934e4cae88a26384ee5129; no voters:
I20260619 18:51:30.997884 2296 leader_election.cc:288] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:30.997994 2306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:30.998241 2306 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [term 1 LEADER]: Becoming Leader. State: Replica: fe8e85700d934e4cae88a26384ee5129, State: Running, Role: LEADER
I20260619 18:51:30.998580 2306 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8e85700d934e4cae88a26384ee5129" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } }
I20260619 18:51:30.999095 2296 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:31.000264 2307 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe8e85700d934e4cae88a26384ee5129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8e85700d934e4cae88a26384ee5129" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } } }
I20260619 18:51:31.000264 2308 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe8e85700d934e4cae88a26384ee5129. Latest consensus state: current_term: 1 leader_uuid: "fe8e85700d934e4cae88a26384ee5129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8e85700d934e4cae88a26384ee5129" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } } }
I20260619 18:51:31.000418 2308 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:31.000418 2307 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:31.000896 2316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:31.004825 2316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:31.011483 2316 catalog_manager.cc:1347] Generated new cluster ID: 147fac21cde347e58685138d715842e1
I20260619 18:51:31.011564 2316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:31.022421 2316 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:31.022755 2316 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:31.033977 2316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe8e85700d934e4cae88a26384ee5129: Generated new TSK 0
I20260619 18:51:31.034857 2316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:31.038610 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 2228
2026-06-19T18:51:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:31 dist-test-slave-fljv krb5kdc[2327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:31 dist-test-slave-fljv krb5kdc[2327](info): set up 2 sockets
Jun 19 18:51:31 dist-test-slave-fljv krb5kdc[2327](info): commencing operation
krb5kdc: starting...
W20260619 18:51:31.998245 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:32 dist-test-slave-fljv krb5kdc[2327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:32.117049 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:34371
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:35681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:34371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:32.228735 2340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:32.229046 2340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:32.229110 2340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:32.233353 2340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:32.233438 2340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:32.233466 2340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:32.233487 2340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:32.233510 2340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:32.239171 2340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:35681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:34371
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:34371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:32.240327 2340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:32.241443 2340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:32.250058 2348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:32.250056 2345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:32.250056 2346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:32.250098 2340 server_base.cc:1034] running on GCE node
I20260619 18:51:32.250929 2340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:32.252072 2340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:32.253255 2340 hybrid_clock.cc:648] HybridClock initialized: now 1781895092253232 us; error 41 us; skew 500 ppm
Jun 19 18:51:32 dist-test-slave-fljv krb5kdc[2327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895092, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:32.256122 2340 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:32.257117 2340 webserver.cc:460] Webserver started at http://127.12.234.254:38693/ using document root <none> and password file <none>
I20260619 18:51:32.257678 2340 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:32.257740 2340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:32.257946 2340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:32.260139 2340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "6996c0910fd04ab296bf0156b022e676"
format_stamp: "Formatted at 2026-06-19 18:51:32 on dist-test-slave-fljv"
server_key: "a6c666e5943a3351955a2d3708461655"
server_key_iv: "5f5b1204495985668a3b0756bdd57db3"
server_key_version: "encryptionkey@0"
I20260619 18:51:32.260752 2340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "6996c0910fd04ab296bf0156b022e676"
format_stamp: "Formatted at 2026-06-19 18:51:32 on dist-test-slave-fljv"
server_key: "a6c666e5943a3351955a2d3708461655"
server_key_iv: "5f5b1204495985668a3b0756bdd57db3"
server_key_version: "encryptionkey@0"
I20260619 18:51:32.264755 2340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:51:32.267663 2355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:32.269068 2340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 18:51:32.269264 2340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "6996c0910fd04ab296bf0156b022e676"
format_stamp: "Formatted at 2026-06-19 18:51:32 on dist-test-slave-fljv"
server_key: "a6c666e5943a3351955a2d3708461655"
server_key_iv: "5f5b1204495985668a3b0756bdd57db3"
server_key_version: "encryptionkey@0"
I20260619 18:51:32.269393 2340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:32.294602 2340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:32.297533 2340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:32.297714 2340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:32.304718 2340 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:34371
I20260619 18:51:32.304741 2407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:34371 every 8 connection(s)
I20260619 18:51:32.305670 2340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:32.308619 2408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:32.313755 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 2340
I20260619 18:51:32.313902 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:32.314265 13227 external_mini_cluster.cc:1428] Setting key 8cec4ccfbe10197bbf70071d226c3c7f
I20260619 18:51:32.315107 2408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676: Bootstrap starting.
I20260619 18:51:32.317844 2408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:32.318655 2408 log.cc:824] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:32.320536 2408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676: No bootstrap required, opened a new log
Jun 19 18:51:32 dist-test-slave-fljv krb5kdc[2327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:32.323961 2408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6996c0910fd04ab296bf0156b022e676" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34371 } }
I20260619 18:51:32.324195 2408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:32.324254 2408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6996c0910fd04ab296bf0156b022e676, State: Initialized, Role: FOLLOWER
I20260619 18:51:32.324688 2408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6996c0910fd04ab296bf0156b022e676" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34371 } }
I20260619 18:51:32.324819 2408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:32.324883 2408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:32.324982 2408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:32.325871 2408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6996c0910fd04ab296bf0156b022e676" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34371 } }
I20260619 18:51:32.326256 2408 leader_election.cc:302] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6996c0910fd04ab296bf0156b022e676; no voters:
I20260619 18:51:32.326596 2408 leader_election.cc:288] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:32.326689 2413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:32.326897 2413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [term 1 LEADER]: Becoming Leader. State: Replica: 6996c0910fd04ab296bf0156b022e676, State: Running, Role: LEADER
I20260619 18:51:32.327260 2413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6996c0910fd04ab296bf0156b022e676" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34371 } }
I20260619 18:51:32.328087 2408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:32.328985 2415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6996c0910fd04ab296bf0156b022e676. Latest consensus state: current_term: 1 leader_uuid: "6996c0910fd04ab296bf0156b022e676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6996c0910fd04ab296bf0156b022e676" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34371 } } }
I20260619 18:51:32.329108 2415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:32.329288 2411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:32.315874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34013 (local address 127.12.234.254:34371)
0619 18:51:32.316342 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:32.316358 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:32.316576 (+ 218us) server_negotiation.cc:407] Connection header received
0619 18:51:32.317330 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:32.317348 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:32.317728 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:32.318050 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:32.318979 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:32.320004 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:32.320891 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:32.321365 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:32.321535 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:32.323928 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:32.323933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:32.323947 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:32.323976 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:32.325689 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:32.326238 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:32.326243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:32.326248 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:32.326368 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:32.326693 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:32.326697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:32.326699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:32.327230 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:32.327445 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:32.327953 (+ 508us) server_negotiation.cc:299] Negotiation successful
0619 18:51:32.328213 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":314,"thread_start_us":159,"threads_started":1}
I20260619 18:51:32.329461 2414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6996c0910fd04ab296bf0156b022e676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6996c0910fd04ab296bf0156b022e676" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 34371 } } }
I20260619 18:51:32.329542 2414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:32.332587 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 18:51:32.333247 2428 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:51:32.333320 2428 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:32.333410 2422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:32.334475 2422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:32.341033 2422 catalog_manager.cc:1347] Generated new cluster ID: fde0ea0e913148e99def452f08fe2cb7
I20260619 18:51:32.341104 2422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:32.343353 2411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:32.334832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50981 (local address 127.12.234.254:34371)
0619 18:51:32.334945 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:32.334951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:32.335145 (+ 194us) server_negotiation.cc:407] Connection header received
0619 18:51:32.335272 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:32.335275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:32.335320 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:32.335380 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:32.336016 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:32.336552 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:32.337675 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:32.338039 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:32.338154 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:32.340025 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:32.340029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:32.340033 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:32.340062 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:32.341670 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:32.342133 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:32.342136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:32.342137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:32.342245 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:32.342542 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:32.342545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:32.342547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:32.342722 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:32.342848 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:32.343107 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 18:51:32.343199 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 18:51:32.360251 2422 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:32.361212 2422 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:32.367347 2422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6996c0910fd04ab296bf0156b022e676: Generated new TSK 0
I20260619 18:51:32.368130 2422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:32.372002 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 2340
W20260619 18:51:32.376627 2325 connection.cc:535] client connection to 127.12.234.254:34371 recv error: Network error: failed to read from TLS socket (remote: 127.12.234.254:34371): Connection reset by peer (error 104)
2026-06-19T18:51:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:32 dist-test-slave-fljv krb5kdc[2439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:32 dist-test-slave-fljv krb5kdc[2439](info): set up 2 sockets
Jun 19 18:51:32 dist-test-slave-fljv krb5kdc[2439](info): commencing operation
krb5kdc: starting...
W20260619 18:51:33.326117 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:33 dist-test-slave-fljv krb5kdc[2439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:33.444690 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:43455
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:32835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:43455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:33.562517 2452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:33.562824 2452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:33.562868 2452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:33.567015 2452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:33.567095 2452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:33.567114 2452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:33.567130 2452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:33.567144 2452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:33.572804 2452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:32835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:43455
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:43455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:33.573904 2452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:33.574981 2452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:33.583678 2460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:33.583680 2457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:33.583693 2458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:33.583775 2452 server_base.cc:1034] running on GCE node
I20260619 18:51:33.584517 2452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:33.585678 2452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:33.586848 2452 hybrid_clock.cc:648] HybridClock initialized: now 1781895093586811 us; error 52 us; skew 500 ppm
Jun 19 18:51:33 dist-test-slave-fljv krb5kdc[2439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895093, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:33.589646 2452 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:33.590641 2452 webserver.cc:460] Webserver started at http://127.12.234.254:42249/ using document root <none> and password file <none>
I20260619 18:51:33.591204 2452 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:33.591269 2452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:33.591449 2452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:33.593607 2452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "97ffc21eeb1443439af2b2fbf7fb80ea"
format_stamp: "Formatted at 2026-06-19 18:51:33 on dist-test-slave-fljv"
server_key: "ed89ba1c65f37d64313e0d9bd8ab908e"
server_key_iv: "4fced06662811e3679920a04783d68cc"
server_key_version: "encryptionkey@0"
I20260619 18:51:33.594255 2452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "97ffc21eeb1443439af2b2fbf7fb80ea"
format_stamp: "Formatted at 2026-06-19 18:51:33 on dist-test-slave-fljv"
server_key: "ed89ba1c65f37d64313e0d9bd8ab908e"
server_key_iv: "4fced06662811e3679920a04783d68cc"
server_key_version: "encryptionkey@0"
I20260619 18:51:33.598250 2452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:33.600734 2467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:33.602001 2452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 18:51:33.602113 2452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "97ffc21eeb1443439af2b2fbf7fb80ea"
format_stamp: "Formatted at 2026-06-19 18:51:33 on dist-test-slave-fljv"
server_key: "ed89ba1c65f37d64313e0d9bd8ab908e"
server_key_iv: "4fced06662811e3679920a04783d68cc"
server_key_version: "encryptionkey@0"
I20260619 18:51:33.602258 2452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:33.612015 2452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:33.615185 2452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:33.615365 2452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:33.622259 2452 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:43455
I20260619 18:51:33.622272 2519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:43455 every 8 connection(s)
I20260619 18:51:33.623266 2452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:33.626452 2520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:33.631174 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 2452
I20260619 18:51:33.631314 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:33.631639 13227 external_mini_cluster.cc:1428] Setting key c7a390364fd9574e1b1427b1f281baa4
I20260619 18:51:33.633570 2520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea: Bootstrap starting.
I20260619 18:51:33.636523 2520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:33.637445 2520 log.cc:824] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:33.639789 2520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea: No bootstrap required, opened a new log
I20260619 18:51:33.642608 2523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:33.633210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37579 (local address 127.12.234.254:43455)
0619 18:51:33.633697 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:33.633728 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:33.633773 (+ 45us) server_negotiation.cc:407] Connection header received
0619 18:51:33.634709 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:33.634735 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:33.635228 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:33.635618 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:33.636084 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:33.637547 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:33.638759 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:33.639425 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:33.639582 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:33.640521 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:33.640528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:33.640545 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:33.641075 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:33.641151 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:33.641181 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:33.641312 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:33.641440 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 18:51:33.641584 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":157,"threads_started":1}
I20260619 18:51:33.644088 2520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ffc21eeb1443439af2b2fbf7fb80ea" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43455 } }
I20260619 18:51:33.644375 2520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:33.644484 2520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97ffc21eeb1443439af2b2fbf7fb80ea, State: Initialized, Role: FOLLOWER
I20260619 18:51:33.644953 2520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ffc21eeb1443439af2b2fbf7fb80ea" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43455 } }
I20260619 18:51:33.645094 2520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:33.645160 2520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:33.645263 2520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:33.645619 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:33.646445 2520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ffc21eeb1443439af2b2fbf7fb80ea" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43455 } }
I20260619 18:51:33.646821 2520 leader_election.cc:302] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97ffc21eeb1443439af2b2fbf7fb80ea; no voters:
I20260619 18:51:33.647213 2520 leader_election.cc:288] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:33.647310 2529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:33.647522 2529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [term 1 LEADER]: Becoming Leader. State: Replica: 97ffc21eeb1443439af2b2fbf7fb80ea, State: Running, Role: LEADER
I20260619 18:51:33.648355 2529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ffc21eeb1443439af2b2fbf7fb80ea" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43455 } }
I20260619 18:51:33.648483 2520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:33.649582 2529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97ffc21eeb1443439af2b2fbf7fb80ea. Latest consensus state: current_term: 1 leader_uuid: "97ffc21eeb1443439af2b2fbf7fb80ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ffc21eeb1443439af2b2fbf7fb80ea" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43455 } } }
I20260619 18:51:33.649758 2529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:33.650007 2530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97ffc21eeb1443439af2b2fbf7fb80ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ffc21eeb1443439af2b2fbf7fb80ea" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43455 } } }
I20260619 18:51:33.650112 2530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:33.652910 2523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:33.647839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35285 (local address 127.12.234.254:43455)
0619 18:51:33.647965 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:33.647969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:33.648251 (+ 282us) server_negotiation.cc:407] Connection header received
0619 18:51:33.648288 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:33.648290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:33.648336 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:33.648409 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:33.648822 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:33.649592 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:33.650787 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:33.651106 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:33.651210 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:33.652430 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:33.652434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:33.652437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:33.652450 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:33.652477 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:33.652490 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:33.652570 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:33.652746 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 18:51:33.652784 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 18:51:33.653649 2545 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:51:33.653730 2545 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:33.653806 2540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:33.654631 2540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:33.660092 2540 catalog_manager.cc:1347] Generated new cluster ID: 32c2e4c1a0e84a2a87b5d431aefaf5db
I20260619 18:51:33.660168 2540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:33.682416 2540 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:33.683454 2540 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:33.690392 2540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97ffc21eeb1443439af2b2fbf7fb80ea: Generated new TSK 0
I20260619 18:51:33.691210 2540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:33.698702 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 2452
W20260619 18:51:33.703599 2437 connection.cc:535] client connection to 127.12.234.254:43455 recv error: Network error: failed to read from TLS socket (remote: 127.12.234.254:43455): Connection reset by peer (error 104)
2026-06-19T18:51:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:33 dist-test-slave-fljv krb5kdc[2551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:33 dist-test-slave-fljv krb5kdc[2551](info): set up 2 sockets
Jun 19 18:51:33 dist-test-slave-fljv krb5kdc[2551](info): commencing operation
krb5kdc: starting...
W20260619 18:51:34.658105 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:34 dist-test-slave-fljv krb5kdc[2551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:34.776873 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:38821
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:38821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:34.889700 2564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:34.890009 2564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:34.890116 2564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:34.894470 2564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:34.894536 2564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:34.894554 2564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:34.894583 2564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:34.894610 2564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:34.900029 2564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:38821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:38821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:34.901093 2564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:34.902144 2564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:51:34.910735 2564 server_base.cc:1034] running on GCE node
W20260619 18:51:34.910701 2572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:34.910655 2570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:34.910650 2569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:34.911576 2564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:34.912791 2564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:34.913993 2564 hybrid_clock.cc:648] HybridClock initialized: now 1781895094913970 us; error 36 us; skew 500 ppm
Jun 19 18:51:34 dist-test-slave-fljv krb5kdc[2551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895094, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:34.916939 2564 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:34.917955 2564 webserver.cc:460] Webserver started at http://127.12.234.254:38525/ using document root <none> and password file <none>
I20260619 18:51:34.918569 2564 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:34.918635 2564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:34.918839 2564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:34.920957 2564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "c01ac898b4754dc7b60c30847c78694b"
format_stamp: "Formatted at 2026-06-19 18:51:34 on dist-test-slave-fljv"
server_key: "17b345b274bf0a45316282b484c17f35"
server_key_iv: "acff51b106e701011270563dd631d2e9"
server_key_version: "encryptionkey@0"
I20260619 18:51:34.921531 2564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "c01ac898b4754dc7b60c30847c78694b"
format_stamp: "Formatted at 2026-06-19 18:51:34 on dist-test-slave-fljv"
server_key: "17b345b274bf0a45316282b484c17f35"
server_key_iv: "acff51b106e701011270563dd631d2e9"
server_key_version: "encryptionkey@0"
I20260619 18:51:34.925520 2564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:51:34.928033 2579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:34.929278 2564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 18:51:34.929404 2564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "c01ac898b4754dc7b60c30847c78694b"
format_stamp: "Formatted at 2026-06-19 18:51:34 on dist-test-slave-fljv"
server_key: "17b345b274bf0a45316282b484c17f35"
server_key_iv: "acff51b106e701011270563dd631d2e9"
server_key_version: "encryptionkey@0"
I20260619 18:51:34.929505 2564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:34.934149 2564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:34.937250 2564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:34.937399 2564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:34.944096 2564 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:38821
I20260619 18:51:34.944103 2631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:38821 every 8 connection(s)
I20260619 18:51:34.945060 2564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:34.947868 2632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:34.953142 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 2564
I20260619 18:51:34.953260 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:34.953500 13227 external_mini_cluster.cc:1428] Setting key 3d996f985e95206f1b48a89eaeeb551f
I20260619 18:51:34.953764 2632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b: Bootstrap starting.
I20260619 18:51:34.956432 2632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:34.957321 2632 log.cc:824] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:34.958988 2635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:34.954922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43455 (local address 127.12.234.254:38821)
0619 18:51:34.955360 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:34.955373 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:34.955412 (+ 39us) server_negotiation.cc:407] Connection header received
0619 18:51:34.956097 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:34.956115 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:34.956439 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:34.956767 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:34.957641 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:34.957646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:34.957657 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:34.957669 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:34.957729 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:34.957749 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:34.957846 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:34.957978 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 18:51:34.958143 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
I20260619 18:51:34.959726 2632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b: No bootstrap required, opened a new log
I20260619 18:51:34.963968 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:34.964236 2632 raft_consensus.cc:348] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ac898b4754dc7b60c30847c78694b" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38821 } }
I20260619 18:51:34.964483 2632 raft_consensus.cc:374] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:34.964537 2632 raft_consensus.cc:729] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01ac898b4754dc7b60c30847c78694b, State: Initialized, Role: FOLLOWER
I20260619 18:51:34.964987 2632 consensus_queue.cc:260] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ac898b4754dc7b60c30847c78694b" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38821 } }
I20260619 18:51:34.965126 2632 raft_consensus.cc:388] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:34.965193 2632 raft_consensus.cc:482] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:34.965312 2632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:34.966457 2632 raft_consensus.cc:504] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ac898b4754dc7b60c30847c78694b" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38821 } }
I20260619 18:51:34.966873 2632 leader_election.cc:302] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c01ac898b4754dc7b60c30847c78694b; no voters:
I20260619 18:51:34.967237 2632 leader_election.cc:288] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:34.967365 2642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:34.967578 2642 raft_consensus.cc:686] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [term 1 LEADER]: Becoming Leader. State: Replica: c01ac898b4754dc7b60c30847c78694b, State: Running, Role: LEADER
I20260619 18:51:34.967909 2642 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ac898b4754dc7b60c30847c78694b" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38821 } }
I20260619 18:51:34.968714 2632 sys_catalog.cc:564] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:34.968909 2635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:34.966266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51643 (local address 127.12.234.254:38821)
0619 18:51:34.966447 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:34.966452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:34.966718 (+ 266us) server_negotiation.cc:407] Connection header received
0619 18:51:34.966881 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:34.966884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:34.966926 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:34.966998 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:34.967931 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:34.967936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:34.967939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:34.967952 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:34.967984 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:34.968024 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:34.968242 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:34.968637 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 18:51:34.968679 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 18:51:34.969672 2644 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c01ac898b4754dc7b60c30847c78694b. Latest consensus state: current_term: 1 leader_uuid: "c01ac898b4754dc7b60c30847c78694b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ac898b4754dc7b60c30847c78694b" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38821 } } }
I20260619 18:51:34.969799 2644 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:34.969903 2643 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c01ac898b4754dc7b60c30847c78694b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ac898b4754dc7b60c30847c78694b" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 38821 } } }
I20260619 18:51:34.969974 2643 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:34.970239 2651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:34.973663 2651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:34.979976 2651 catalog_manager.cc:1347] Generated new cluster ID: c020e341f63c4375a5e149301655f0e7
I20260619 18:51:34.980044 2651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:34.989534 2651 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:34.989836 2651 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:34.996374 2651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c01ac898b4754dc7b60c30847c78694b: Generated new TSK 0
I20260619 18:51:34.997164 2651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:35.003494 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 2564
2026-06-19T18:51:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:35 dist-test-slave-fljv krb5kdc[2663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:35 dist-test-slave-fljv krb5kdc[2663](info): set up 2 sockets
Jun 19 18:51:35 dist-test-slave-fljv krb5kdc[2663](info): commencing operation
krb5kdc: starting...
W20260619 18:51:35.962340 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:36 dist-test-slave-fljv krb5kdc[2663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:36.081219 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46583
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:46583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:36.190541 2676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:36.190836 2676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:36.190879 2676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:36.195010 2676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:36.195096 2676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:36.195120 2676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:36.195142 2676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:36.195155 2676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:36.200734 2676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:46583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:46583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:36.201869 2676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:36.202944 2676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:36.211581 2681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:36.211591 2684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:36.211681 2682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:36.211656 2676 server_base.cc:1034] running on GCE node
I20260619 18:51:36.212373 2676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:36.213508 2676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:36.214702 2676 hybrid_clock.cc:648] HybridClock initialized: now 1781895096214680 us; error 35 us; skew 500 ppm
Jun 19 18:51:36 dist-test-slave-fljv krb5kdc[2663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895096, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:36.222827 2676 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:36.223886 2676 webserver.cc:460] Webserver started at http://127.12.234.254:39469/ using document root <none> and password file <none>
I20260619 18:51:36.224454 2676 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:36.224525 2676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:36.224772 2676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:36.226934 2676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "7884c3d80adc478a97170b2e7367a37d"
format_stamp: "Formatted at 2026-06-19 18:51:36 on dist-test-slave-fljv"
server_key: "042b2eb6d0f67cba27c623ea4213e81e"
server_key_iv: "fe540000240ecc24634b1ea51d364fa6"
server_key_version: "encryptionkey@0"
I20260619 18:51:36.227481 2676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "7884c3d80adc478a97170b2e7367a37d"
format_stamp: "Formatted at 2026-06-19 18:51:36 on dist-test-slave-fljv"
server_key: "042b2eb6d0f67cba27c623ea4213e81e"
server_key_iv: "fe540000240ecc24634b1ea51d364fa6"
server_key_version: "encryptionkey@0"
I20260619 18:51:36.231385 2676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 18:51:36.233948 2691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:36.235301 2676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 18:51:36.235443 2676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "7884c3d80adc478a97170b2e7367a37d"
format_stamp: "Formatted at 2026-06-19 18:51:36 on dist-test-slave-fljv"
server_key: "042b2eb6d0f67cba27c623ea4213e81e"
server_key_iv: "fe540000240ecc24634b1ea51d364fa6"
server_key_version: "encryptionkey@0"
I20260619 18:51:36.235558 2676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:36.263983 2676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:36.267052 2676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:36.267238 2676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:36.274281 2676 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:46583
I20260619 18:51:36.274305 2743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:46583 every 8 connection(s)
I20260619 18:51:36.275389 2676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:36.277784 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 2676
I20260619 18:51:36.277920 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:36.278276 13227 external_mini_cluster.cc:1428] Setting key 2e01049cfadc56900dec09c06839c234
I20260619 18:51:36.278590 2744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:36.285434 2744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d: Bootstrap starting.
Jun 19 18:51:36 dist-test-slave-fljv krb5kdc[2663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:36.288050 2744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:36.288780 2744 log.cc:824] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:36.291036 2744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d: No bootstrap required, opened a new log
I20260619 18:51:36.293087 2747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:36.279634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46141 (local address 127.12.234.254:46583)
0619 18:51:36.280056 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:36.280072 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:36.280108 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:51:36.280753 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:36.280780 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:36.281130 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:36.281404 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:36.282304 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:36.283309 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:36.284511 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:36.284996 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:36.285125 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:36.287971 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:36.287976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:36.287990 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:36.288019 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:36.289759 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:36.290170 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:36.290200 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:36.290207 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:36.290346 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:36.290582 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:36.290585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:36.290586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:36.291099 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:36.291286 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:36.291531 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 18:51:36.291841 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":121,"threads_started":1}
I20260619 18:51:36.294852 2744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7884c3d80adc478a97170b2e7367a37d" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46583 } }
I20260619 18:51:36.295126 2744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:36.295192 2744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7884c3d80adc478a97170b2e7367a37d, State: Initialized, Role: FOLLOWER
I20260619 18:51:36.295647 2744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7884c3d80adc478a97170b2e7367a37d" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46583 } }
I20260619 18:51:36.295791 2744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:36.295854 2744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:36.295953 2744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:36.296033 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:36.296859 2744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7884c3d80adc478a97170b2e7367a37d" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46583 } }
I20260619 18:51:36.297219 2744 leader_election.cc:302] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7884c3d80adc478a97170b2e7367a37d; no voters:
I20260619 18:51:36.297565 2744 leader_election.cc:288] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:36.297640 2753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:36.297842 2753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [term 1 LEADER]: Becoming Leader. State: Replica: 7884c3d80adc478a97170b2e7367a37d, State: Running, Role: LEADER
I20260619 18:51:36.298138 2753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7884c3d80adc478a97170b2e7367a37d" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46583 } }
I20260619 18:51:36.298879 2744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:36.299733 2754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7884c3d80adc478a97170b2e7367a37d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7884c3d80adc478a97170b2e7367a37d" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46583 } } }
I20260619 18:51:36.299868 2754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:36.300083 2754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7884c3d80adc478a97170b2e7367a37d. Latest consensus state: current_term: 1 leader_uuid: "7884c3d80adc478a97170b2e7367a37d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7884c3d80adc478a97170b2e7367a37d" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 46583 } } }
I20260619 18:51:36.300179 2754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:36.300457 2763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:36.304051 2763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:36.306576 2747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:36.297957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46355 (local address 127.12.234.254:46583)
0619 18:51:36.298058 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:36.298063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:36.298523 (+ 460us) server_negotiation.cc:407] Connection header received
0619 18:51:36.298625 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:36.298629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:36.298679 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:36.298740 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:36.299480 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:36.300254 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:36.301367 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:36.301602 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:36.301694 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:36.303156 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:36.303161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:36.303164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:36.303190 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:36.304753 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:36.305232 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:36.305236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:36.305237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:36.305323 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:36.305611 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:36.305615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:36.305617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:36.305850 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:36.306023 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:36.306296 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 18:51:36.306416 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":37}
I20260619 18:51:36.309909 2763 catalog_manager.cc:1347] Generated new cluster ID: b8cadf4fb57140379f0f66fc02e4fedf
I20260619 18:51:36.309979 2763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:36.326956 2763 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:36.327930 2763 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:36.336421 2763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7884c3d80adc478a97170b2e7367a37d: Generated new TSK 0
I20260619 18:51:36.337174 2763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:36.342056 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 2676
2026-06-19T18:51:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:36 dist-test-slave-fljv krb5kdc[2775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:36 dist-test-slave-fljv krb5kdc[2775](info): set up 2 sockets
Jun 19 18:51:36 dist-test-slave-fljv krb5kdc[2775](info): commencing operation
krb5kdc: starting...
W20260619 18:51:37.306313 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:37 dist-test-slave-fljv krb5kdc[2775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:37.429560 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41973
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:41973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:37.543720 2788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:37.544023 2788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:37.544068 2788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:37.548259 2788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:37.548336 2788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:37.548354 2788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:37.548370 2788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:37.548385 2788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:37.553929 2788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:41973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:41973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:37.555191 2788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:37.556293 2788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:37.565385 2796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:37.565424 2793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:37.565414 2794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:37.565622 2788 server_base.cc:1034] running on GCE node
I20260619 18:51:37.566288 2788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:37.567481 2788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:37.568665 2788 hybrid_clock.cc:648] HybridClock initialized: now 1781895097568641 us; error 39 us; skew 500 ppm
Jun 19 18:51:37 dist-test-slave-fljv krb5kdc[2775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895097, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:37.571520 2788 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:37.572454 2788 webserver.cc:460] Webserver started at http://127.12.234.254:39697/ using document root <none> and password file <none>
I20260619 18:51:37.572984 2788 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:37.573042 2788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:37.573225 2788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:37.575352 2788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "047a8bfeed2f4517818e562dc81b0101"
format_stamp: "Formatted at 2026-06-19 18:51:37 on dist-test-slave-fljv"
server_key: "a1a3de827bbd21c5f06ce86cfc488b99"
server_key_iv: "093306f4f256979f1d8f653486eaf66c"
server_key_version: "encryptionkey@0"
I20260619 18:51:37.575908 2788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "047a8bfeed2f4517818e562dc81b0101"
format_stamp: "Formatted at 2026-06-19 18:51:37 on dist-test-slave-fljv"
server_key: "a1a3de827bbd21c5f06ce86cfc488b99"
server_key_iv: "093306f4f256979f1d8f653486eaf66c"
server_key_version: "encryptionkey@0"
I20260619 18:51:37.579797 2788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 18:51:37.582307 2803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:37.583675 2788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 18:51:37.583837 2788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "047a8bfeed2f4517818e562dc81b0101"
format_stamp: "Formatted at 2026-06-19 18:51:37 on dist-test-slave-fljv"
server_key: "a1a3de827bbd21c5f06ce86cfc488b99"
server_key_iv: "093306f4f256979f1d8f653486eaf66c"
server_key_version: "encryptionkey@0"
I20260619 18:51:37.583966 2788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:37.601291 2788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:37.604611 2788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:37.604827 2788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:37.612160 2788 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:41973
I20260619 18:51:37.612223 2855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:41973 every 8 connection(s)
I20260619 18:51:37.613229 2788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:37.616285 2856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:37.616355 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 2788
I20260619 18:51:37.616489 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:37.616779 13227 external_mini_cluster.cc:1428] Setting key 8b89f4a851970befda46c246d662a1b3
I20260619 18:51:37.623489 2856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101: Bootstrap starting.
I20260619 18:51:37.625836 2856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:37.626623 2856 log.cc:824] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:37.626926 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:37.618237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46247 (local address 127.12.234.254:41973)
0619 18:51:37.618758 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:37.618774 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:37.618815 (+ 41us) server_negotiation.cc:407] Connection header received
0619 18:51:37.619959 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:37.619990 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:37.620476 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:37.620882 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:37.621370 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:37.622501 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:37.623475 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:37.624011 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:37.624172 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:37.625014 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:37.625019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:37.625032 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:37.625471 (+ 439us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:37.625538 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:37.625564 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:37.625655 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:37.625812 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 18:51:37.626052 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":167,"threads_started":1}
I20260619 18:51:37.628872 2856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101: No bootstrap required, opened a new log
I20260619 18:51:37.630038 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:37.632911 2856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a8bfeed2f4517818e562dc81b0101" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41973 } }
I20260619 18:51:37.633188 2856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:37.633258 2856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047a8bfeed2f4517818e562dc81b0101, State: Initialized, Role: FOLLOWER
I20260619 18:51:37.633767 2856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a8bfeed2f4517818e562dc81b0101" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41973 } }
I20260619 18:51:37.633956 2856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:37.634028 2856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:37.634158 2856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:37.635393 2856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a8bfeed2f4517818e562dc81b0101" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41973 } }
I20260619 18:51:37.635840 2856 leader_election.cc:302] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 047a8bfeed2f4517818e562dc81b0101; no voters:
I20260619 18:51:37.636307 2856 leader_election.cc:288] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:37.636466 2866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:37.636749 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:37.632031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48401 (local address 127.12.234.254:41973)
0619 18:51:37.632151 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:37.632156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:37.632335 (+ 179us) server_negotiation.cc:407] Connection header received
0619 18:51:37.632464 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:37.632467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:37.632511 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:37.632572 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:37.632877 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:37.633466 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:37.634681 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:37.635044 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:37.635145 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:37.636098 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:37.636102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:37.636104 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:37.636124 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:37.636161 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:37.636177 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:37.636394 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:37.636491 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 18:51:37.636564 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 18:51:37.636749 2866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [term 1 LEADER]: Becoming Leader. State: Replica: 047a8bfeed2f4517818e562dc81b0101, State: Running, Role: LEADER
I20260619 18:51:37.637171 2866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a8bfeed2f4517818e562dc81b0101" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41973 } }
I20260619 18:51:37.638605 2868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 047a8bfeed2f4517818e562dc81b0101. Latest consensus state: current_term: 1 leader_uuid: "047a8bfeed2f4517818e562dc81b0101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a8bfeed2f4517818e562dc81b0101" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41973 } } }
I20260619 18:51:37.638727 2856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:37.638792 2868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:37.638413 2866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "047a8bfeed2f4517818e562dc81b0101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a8bfeed2f4517818e562dc81b0101" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 41973 } } }
I20260619 18:51:37.639027 2866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:37.639340 2870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:37.643642 2870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:37.649818 2870 catalog_manager.cc:1347] Generated new cluster ID: 3f85fdb69cae4b57a449e505916d30a0
I20260619 18:51:37.649897 2870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:37.674019 2870 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:37.675171 2870 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:37.682777 2870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 047a8bfeed2f4517818e562dc81b0101: Generated new TSK 0
I20260619 18:51:37.683540 2870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:37.692775 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 2788
2026-06-19T18:51:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:37 dist-test-slave-fljv krb5kdc[2887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:37 dist-test-slave-fljv krb5kdc[2887](info): set up 2 sockets
Jun 19 18:51:37 dist-test-slave-fljv krb5kdc[2887](info): commencing operation
krb5kdc: starting...
W20260619 18:51:38.666306 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:38 dist-test-slave-fljv krb5kdc[2887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:38.786665 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:39741
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:39741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:38.901319 2900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:38.901615 2900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:38.901657 2900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:38.905858 2900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:38.905938 2900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:38.905957 2900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:38.905972 2900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:38.905987 2900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:38.911643 2900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:39741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:39741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:38.912779 2900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:38.913852 2900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:51:38.922684 2900 server_base.cc:1034] running on GCE node
W20260619 18:51:38.922640 2906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:38.922640 2908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:38.922665 2905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:38.923478 2900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:38.924660 2900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:38.925834 2900 hybrid_clock.cc:648] HybridClock initialized: now 1781895098925805 us; error 46 us; skew 500 ppm
Jun 19 18:51:38 dist-test-slave-fljv krb5kdc[2887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895098, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:38.928692 2900 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:38.929634 2900 webserver.cc:460] Webserver started at http://127.12.234.254:37691/ using document root <none> and password file <none>
I20260619 18:51:38.930168 2900 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:38.930310 2900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:38.930521 2900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:38.932570 2900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "cf2e0de473904a6fb49f48f5b5218491"
format_stamp: "Formatted at 2026-06-19 18:51:38 on dist-test-slave-fljv"
server_key: "0a9a791d1165559c94f75b0ac06dd85b"
server_key_iv: "392afe8131858ec56caaaec7d80dbd3d"
server_key_version: "encryptionkey@0"
I20260619 18:51:38.933108 2900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "cf2e0de473904a6fb49f48f5b5218491"
format_stamp: "Formatted at 2026-06-19 18:51:38 on dist-test-slave-fljv"
server_key: "0a9a791d1165559c94f75b0ac06dd85b"
server_key_iv: "392afe8131858ec56caaaec7d80dbd3d"
server_key_version: "encryptionkey@0"
I20260619 18:51:38.936925 2900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 18:51:38.939455 2915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:38.940810 2900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 18:51:38.940934 2900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "cf2e0de473904a6fb49f48f5b5218491"
format_stamp: "Formatted at 2026-06-19 18:51:38 on dist-test-slave-fljv"
server_key: "0a9a791d1165559c94f75b0ac06dd85b"
server_key_iv: "392afe8131858ec56caaaec7d80dbd3d"
server_key_version: "encryptionkey@0"
I20260619 18:51:38.941046 2900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:38.945878 2900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:38.949108 2900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:38.949288 2900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:38.956236 2900 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:39741
I20260619 18:51:38.956266 2967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:39741 every 8 connection(s)
I20260619 18:51:38.957245 2900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:38.960495 2968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:38.963014 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 2900
I20260619 18:51:38.963135 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:38.963392 13227 external_mini_cluster.cc:1428] Setting key 20b053373b4f7fb6bedd7120ea47f271
I20260619 18:51:38.966849 2968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491: Bootstrap starting.
I20260619 18:51:38.968883 2971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:38.964886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50913 (local address 127.12.234.254:39741)
0619 18:51:38.965378 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:38.965395 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:38.965435 (+ 40us) server_negotiation.cc:407] Connection header received
0619 18:51:38.966101 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:38.966119 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:38.966492 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:38.966799 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:38.967649 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:38.967654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:38.967667 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:38.967677 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:38.967730 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:38.967749 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:38.967856 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:38.967947 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 18:51:38.968092 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":132,"threads_started":1}
I20260619 18:51:38.969441 2968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:38.970216 2968 log.cc:824] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:38.971716 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:38.972455 2968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491: No bootstrap required, opened a new log
I20260619 18:51:38.976306 2971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:38.974318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57977 (local address 127.12.234.254:39741)
0619 18:51:38.974463 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:38.974468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:38.974818 (+ 350us) server_negotiation.cc:407] Connection header received
0619 18:51:38.975056 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:38.975060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:38.975099 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:38.975194 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:38.975922 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:38.975925 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:38.975928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 18:51:38.975940 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:38.975973 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 18:51:38.975987 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:38.976077 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:38.976138 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 18:51:38.976176 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 18:51:38.976984 2968 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2e0de473904a6fb49f48f5b5218491" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39741 } }
I20260619 18:51:38.977309 2968 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 18:51:38.977375 2968 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2e0de473904a6fb49f48f5b5218491, State: Initialized, Role: FOLLOWER
I20260619 18:51:38.977823 2968 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2e0de473904a6fb49f48f5b5218491" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39741 } }
I20260619 18:51:38.977952 2968 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 18:51:38.977993 2968 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 18:51:38.978065 2968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:38.979151 2968 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2e0de473904a6fb49f48f5b5218491" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39741 } }
I20260619 18:51:38.979518 2968 leader_election.cc:302] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf2e0de473904a6fb49f48f5b5218491; no voters:
I20260619 18:51:38.979842 2968 leader_election.cc:288] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 18:51:38.979959 2978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:38.980221 2978 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [term 1 LEADER]: Becoming Leader. State: Replica: cf2e0de473904a6fb49f48f5b5218491, State: Running, Role: LEADER
I20260619 18:51:38.980589 2978 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2e0de473904a6fb49f48f5b5218491" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39741 } }
I20260619 18:51:38.981178 2968 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:38.982358 2979 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf2e0de473904a6fb49f48f5b5218491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2e0de473904a6fb49f48f5b5218491" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39741 } } }
I20260619 18:51:38.982486 2979 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:38.982758 2980 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf2e0de473904a6fb49f48f5b5218491. Latest consensus state: current_term: 1 leader_uuid: "cf2e0de473904a6fb49f48f5b5218491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2e0de473904a6fb49f48f5b5218491" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 39741 } } }
I20260619 18:51:38.982892 2988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:38.982920 2980 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:38.985934 2988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:38.992496 2988 catalog_manager.cc:1347] Generated new cluster ID: da12ac6e75ab468fa0065b929c9d0773
I20260619 18:51:38.992599 2988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:39.003985 2988 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:39.004355 2988 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:39.015595 2988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf2e0de473904a6fb49f48f5b5218491: Generated new TSK 0
I20260619 18:51:39.016356 2988 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 18:51:39.021593 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 2900
2026-06-19T18:51:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1328 ms)
[----------] 18 tests from AuthTokenIssuingTest (24000 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:39 dist-test-slave-fljv krb5kdc[2999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:39 dist-test-slave-fljv krb5kdc[2999](info): set up 2 sockets
Jun 19 18:51:39 dist-test-slave-fljv krb5kdc[2999](info): commencing operation
krb5kdc: starting...
W20260619 18:51:39.994307 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:40.117856 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:37199
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:38117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:37199,127.12.234.253:33109,127.12.234.252:41829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:40.230738 3012 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 18:51:40.231117 3012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:40.231153 3012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:40.231192 3012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:40.235419 3012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:40.235495 3012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:40.235515 3012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:40.235531 3012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:40.235548 3012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:40.241204 3012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:38117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:37199,127.12.234.253:33109,127.12.234.252:41829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:37199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:40.242353 3012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:40.243369 3012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:40.252115 3017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:40.252096 3020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:40.252183 3012 server_base.cc:1034] running on GCE node
W20260619 18:51:40.252128 3018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:40.252952 3012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:40.254107 3012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:40.255295 3012 hybrid_clock.cc:648] HybridClock initialized: now 1781895100255272 us; error 32 us; skew 500 ppm
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:40.258111 3012 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:40.259120 3012 webserver.cc:460] Webserver started at http://127.12.234.254:37335/ using document root <none> and password file <none>
I20260619 18:51:40.259693 3012 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:40.259757 3012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:40.259961 3012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:40.262013 3012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "c44edec7225c49d38a40ac5e71ce3f65"
format_stamp: "Formatted at 2026-06-19 18:51:40 on dist-test-slave-fljv"
server_key: "eebd4451ee738b6f12fbc3d08956d1bc"
server_key_iv: "f0db4b3adab44ca96b19e65dec1273e3"
server_key_version: "encryptionkey@0"
I20260619 18:51:40.262611 3012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "c44edec7225c49d38a40ac5e71ce3f65"
format_stamp: "Formatted at 2026-06-19 18:51:40 on dist-test-slave-fljv"
server_key: "eebd4451ee738b6f12fbc3d08956d1bc"
server_key_iv: "f0db4b3adab44ca96b19e65dec1273e3"
server_key_version: "encryptionkey@0"
I20260619 18:51:40.266486 3012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:51:40.269057 3027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:40.270465 3012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:40.270622 3012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "c44edec7225c49d38a40ac5e71ce3f65"
format_stamp: "Formatted at 2026-06-19 18:51:40 on dist-test-slave-fljv"
server_key: "eebd4451ee738b6f12fbc3d08956d1bc"
server_key_iv: "f0db4b3adab44ca96b19e65dec1273e3"
server_key_version: "encryptionkey@0"
I20260619 18:51:40.270738 3012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:40.296684 3012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:40.299999 3012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:40.300186 3012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:40.307576 3012 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:37199
I20260619 18:51:40.307830 3079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:37199 every 8 connection(s)
I20260619 18:51:40.308640 3012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:40.311609 3080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:40.314755 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 3012
I20260619 18:51:40.314875 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:40.315197 13227 external_mini_cluster.cc:1428] Setting key c4976e7bc459a14538d1e9faa37cfb96
I20260619 18:51:40.315755 3080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.234.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.253@KRBTEST.COM" created.
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:40.331276 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.318356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:37199 (local address 127.0.0.1:43278)
0619 18:51:40.318980 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.319019 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:40.319921 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:40.320691 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:40.320697 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:40.321147 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:40.321400 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.321414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.322593 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.322595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.323375 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.323385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.324053 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.324056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.324182 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.324734 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:40.324782 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:40.326370 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:40.328193 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.328202 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.328214 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.328435 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.328753 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.328756 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.328758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.328870 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.329470 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:40.329478 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:40.329618 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 18:51:40.329778 (+ 160us) client_negotiation.cc:240] Negotiation successful
0619 18:51:40.330214 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":99,"spinlock_wait_cycles":39168,"thread_start_us":157,"threads_started":1}
I20260619 18:51:40.331276 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.318356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43278 (local address 127.12.234.254:37199)
0619 18:51:40.318835 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.318854 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.319136 (+ 282us) server_negotiation.cc:407] Connection header received
0619 18:51:40.320288 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.320292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.320492 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.320665 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.321548 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.322483 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.323484 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.323904 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.324054 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.326516 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.326521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.326533 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.326554 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.328078 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.328535 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.328540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.328542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.328657 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.328950 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.328953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.328955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.329362 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.329486 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.329852 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.330120 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":320,"thread_start_us":130,"threads_started":1}
I20260619 18:51:40.334789 3080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:40.335769 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.335393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:33109 (local address 127.0.0.1:39968)
0619 18:51:40.335567 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.335682 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260619 18:51:40.336045 3030 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
I20260619 18:51:40.337052 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.336800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:33109 (local address 127.0.0.1:39978)
0619 18:51:40.336914 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.336977 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 18:51:40.337230 3080 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.253:33109: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab.
Entry for principal kudu/127.12.234.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab.
I20260619 18:51:40.341301 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.253:33109
--webserver_interface=127.12.234.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab
--principal=kudu/127.12.234.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:38117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:37199,127.12.234.253:33109,127.12.234.252:41829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:51:40.386494 3080 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } attempt: 1
I20260619 18:51:40.387445 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.387059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:33109 (local address 127.0.0.1:39994)
0619 18:51:40.387239 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.387346 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 18:51:40.388316 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.388067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:33109 (local address 127.0.0.1:40008)
0619 18:51:40.388184 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.388243 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 18:51:40.388468 3080 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.253:33109: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
I20260619 18:51:40.456660 3080 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } attempt: 2
I20260619 18:51:40.457541 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.457183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:33109 (local address 127.0.0.1:40012)
0619 18:51:40.457358 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.457441 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 18:51:40.457593 3087 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 18:51:40.457996 3087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:40.458048 3087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:40.458107 3087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 18:51:40.458416 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.458169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:33109 (local address 127.0.0.1:40024)
0619 18:51:40.458294 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.458350 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 18:51:40.458570 3080 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.253:33109: Network error: Client connection negotiation failed: client connection to 127.12.234.253:33109: connect: Connection refused (error 111)
W20260619 18:51:40.462544 3087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:40.462623 3087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:40.462649 3087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:40.462674 3087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:40.462702 3087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:40.468367 3087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:38117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:37199,127.12.234.253:33109,127.12.234.252:41829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab
--principal=kudu/127.12.234.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.253:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.234.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:40.469532 3087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:40.470659 3087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:40.479887 3092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:40.479893 3093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:40.479995 3087 server_base.cc:1034] running on GCE node
W20260619 18:51:40.479887 3095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:40.480835 3087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:40.482005 3087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:40.483197 3087 hybrid_clock.cc:648] HybridClock initialized: now 1781895100483171 us; error 45 us; skew 500 ppm
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:40.486136 3087 init.cc:377] Logged in from keytab as kudu/127.12.234.253@KRBTEST.COM (short username kudu)
I20260619 18:51:40.487165 3087 webserver.cc:460] Webserver started at http://127.12.234.253:37197/ using document root <none> and password file <none>
I20260619 18:51:40.487738 3087 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:40.487803 3087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:40.487980 3087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:40.490147 3087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/data/instance:
uuid: "8d75be56641741d3ac3a26ba237953b9"
format_stamp: "Formatted at 2026-06-19 18:51:40 on dist-test-slave-fljv"
server_key: "ef354a5a14bf8b75c8ebc5e6a299f5e4"
server_key_iv: "49fa2b98f287abe5111c005591467869"
server_key_version: "encryptionkey@0"
I20260619 18:51:40.490736 3087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/wal/instance:
uuid: "8d75be56641741d3ac3a26ba237953b9"
format_stamp: "Formatted at 2026-06-19 18:51:40 on dist-test-slave-fljv"
server_key: "ef354a5a14bf8b75c8ebc5e6a299f5e4"
server_key_iv: "49fa2b98f287abe5111c005591467869"
server_key_version: "encryptionkey@0"
I20260619 18:51:40.494714 3087 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 18:51:40.497325 3102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:40.498733 3087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:40.498860 3087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/wal
uuid: "8d75be56641741d3ac3a26ba237953b9"
format_stamp: "Formatted at 2026-06-19 18:51:40 on dist-test-slave-fljv"
server_key: "ef354a5a14bf8b75c8ebc5e6a299f5e4"
server_key_iv: "49fa2b98f287abe5111c005591467869"
server_key_version: "encryptionkey@0"
I20260619 18:51:40.498975 3087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:40.508170 3087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:40.511411 3087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:40.511600 3087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:40.518927 3087 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.253:33109
I20260619 18:51:40.519192 3154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.253:33109 every 8 connection(s)
I20260619 18:51:40.519992 3087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
I20260619 18:51:40.522993 3155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:40.526932 3155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:40.528268 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 3087
I20260619 18:51:40.528403 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-1/wal/instance
I20260619 18:51:40.528680 13227 external_mini_cluster.cc:1428] Setting key c51f60703e95a15fe2c1efcc88b3dfce
WARNING: no policy specified for kudu/127.12.234.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.252@KRBTEST.COM" created.
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:40.544930 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.529023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.12.234.254:37199)
0619 18:51:40.529177 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.529182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.530153 (+ 971us) server_negotiation.cc:407] Connection header received
0619 18:51:40.531173 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.531177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.531238 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.531333 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.532610 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.533441 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.534851 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.535219 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.535350 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.540484 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.540487 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.540489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.540519 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.542227 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.542959 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.542963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.542965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.543099 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.543546 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.543549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.543551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.543809 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.543903 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.544677 (+ 774us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.544802 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 18:51:40.546388 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.529460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:37199 (local address 127.0.0.1:43288)
0619 18:51:40.529968 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.530028 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:40.530955 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:40.531516 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:40.531523 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:40.531881 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:40.532437 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.532453 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.533617 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.533620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.534704 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.534716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.535411 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.535415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.535607 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.536245 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:40.536263 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:40.540250 (+ 3987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:40.542358 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.542367 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.542387 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.542779 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.543202 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.543205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.543207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.543421 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.543928 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:40.543936 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:40.544330 (+ 394us) client_negotiation.cc:769] Sending connection context
0619 18:51:40.544639 (+ 309us) client_negotiation.cc:240] Negotiation successful
0619 18:51:40.545072 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":122,"threads_started":1}
I20260619 18:51:40.548127 3155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:40.549759 3080 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } attempt: 3
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab.
Entry for principal kudu/127.12.234.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab.
I20260619 18:51:40.558655 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.548668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:33109 (local address 127.0.0.1:40038)
0619 18:51:40.548813 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.548828 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:40.548899 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:40.549465 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:40.549467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:40.549604 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:40.549756 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.549765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.550972 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.550975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.551672 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.551681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.552385 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.552390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.552500 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.553023 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:40.553037 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:40.554573 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:40.556835 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.556838 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.556840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.557056 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.557336 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.557339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.557341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.557418 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.558002 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:40.558006 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:40.558116 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 18:51:40.558251 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 18:51:40.558462 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":60}
I20260619 18:51:40.558911 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.252:41829
--webserver_interface=127.12.234.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab
--principal=kudu/127.12.234.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:38117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:37199,127.12.234.253:33109,127.12.234.252:41829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:51:40.558866 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.548767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.12.234.253:33109)
0619 18:51:40.549074 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.549087 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.549110 (+ 23us) server_negotiation.cc:407] Connection header received
0619 18:51:40.549154 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.549160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.549350 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.549462 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.549871 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.550866 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.551770 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.552252 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.552343 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.554749 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.554763 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.554783 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.554808 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.556732 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.557163 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.557169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.557171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.557254 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.557492 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.557495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.557497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.557893 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.558043 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.558517 (+ 474us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.558667 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":116,"threads_started":1}
I20260619 18:51:40.560200 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.550255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:33109 (local address 127.0.0.1:40044)
0619 18:51:40.550369 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.550383 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:40.550461 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:40.550764 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:40.550768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:40.550945 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:40.551052 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.551059 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.552010 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.552013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.553230 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.553242 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.553820 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.553824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.553936 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.554433 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:40.554449 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:40.555994 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:40.558629 (+ 2635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.558633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.558635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.558912 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.559245 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.559249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.559251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.559338 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.559686 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:40.559689 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:40.559790 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 18:51:40.559882 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 18:51:40.560035 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 18:51:40.560230 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.550294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40044 (local address 127.12.234.253:33109)
0619 18:51:40.550573 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.550581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.550592 (+ 11us) server_negotiation.cc:407] Connection header received
0619 18:51:40.550617 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.550619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.550650 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.550719 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.551165 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.551879 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.553349 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.553697 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.553782 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.556268 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.556272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.556274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.556299 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.558495 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.559034 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.559036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.559038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.559134 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.559425 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.559428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.559430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.559597 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.559678 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.559938 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.560045 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":212,"thread_start_us":116,"threads_started":1}
I20260619 18:51:40.561100 3155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:40.561687 3080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:40.561822 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.561552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51552)
0619 18:51:40.561678 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.561761 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 18:51:40.562057 3103 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
I20260619 18:51:40.562430 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.562119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51554)
0619 18:51:40.562313 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.562359 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260619 18:51:40.562942 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.562704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51566)
0619 18:51:40.562836 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.562883 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260619 18:51:40.563069 3155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:41829: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
I20260619 18:51:40.563254 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.562977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51570)
0619 18:51:40.563140 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.563187 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260619 18:51:40.563376 3080 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:41829: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
I20260619 18:51:40.594549 3080 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } attempt: 1
I20260619 18:51:40.595451 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.595082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51578)
0619 18:51:40.595265 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.595348 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 18:51:40.596287 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.596040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51584)
0619 18:51:40.596156 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.596216 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 18:51:40.596428 3080 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:41829: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
I20260619 18:51:40.612358 3155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } attempt: 1
I20260619 18:51:40.613296 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.612946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51592)
0619 18:51:40.613109 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.613176 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 18:51:40.613986 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.613783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51600)
0619 18:51:40.613885 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.613930 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 18:51:40.614101 3155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:41829: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
I20260619 18:51:40.671609 3080 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } attempt: 2
I20260619 18:51:40.672580 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.672167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51606)
0619 18:51:40.672340 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.672439 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 18:51:40.673364 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.673150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51608)
0619 18:51:40.673249 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.673300 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 18:51:40.673493 3080 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:41829: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
W20260619 18:51:40.675576 3162 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 18:51:40.675879 3162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:40.675920 3162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:40.675983 3162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:40.680285 3162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:40.680359 3162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:40.680378 3162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:40.680395 3162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:40.680413 3162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:40.682250 3155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } attempt: 2
I20260619 18:51:40.683022 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.682745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51616)
0619 18:51:40.682882 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.682944 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 18:51:40.683818 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.683597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51618)
0619 18:51:40.683696 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.683759 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 18:51:40.683946 3155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:41829: Network error: Client connection negotiation failed: client connection to 127.12.234.252:41829: connect: Connection refused (error 111)
I20260619 18:51:40.686417 3162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:38117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:37199,127.12.234.253:33109,127.12.234.252:41829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab
--principal=kudu/127.12.234.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.252:41829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.234.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:40.687515 3162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:40.688530 3162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:40.697425 3169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:40.697489 3162 server_base.cc:1034] running on GCE node
W20260619 18:51:40.697428 3168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:40.697763 3171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:40.698345 3162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:40.699486 3162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:40.700641 3162 hybrid_clock.cc:648] HybridClock initialized: now 1781895100700624 us; error 35 us; skew 500 ppm
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:40.703544 3162 init.cc:377] Logged in from keytab as kudu/127.12.234.252@KRBTEST.COM (short username kudu)
I20260619 18:51:40.704533 3162 webserver.cc:460] Webserver started at http://127.12.234.252:41025/ using document root <none> and password file <none>
I20260619 18:51:40.705163 3162 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:40.705231 3162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:40.705407 3162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:40.707649 3162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/data/instance:
uuid: "a52cbd74ae76452d841b7d370265a5b4"
format_stamp: "Formatted at 2026-06-19 18:51:40 on dist-test-slave-fljv"
server_key: "440cc9af924651bc89f6f5da48baac92"
server_key_iv: "012e281e7a3188e658ab6b6a8e80a057"
server_key_version: "encryptionkey@0"
I20260619 18:51:40.708186 3162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/wal/instance:
uuid: "a52cbd74ae76452d841b7d370265a5b4"
format_stamp: "Formatted at 2026-06-19 18:51:40 on dist-test-slave-fljv"
server_key: "440cc9af924651bc89f6f5da48baac92"
server_key_iv: "012e281e7a3188e658ab6b6a8e80a057"
server_key_version: "encryptionkey@0"
I20260619 18:51:40.712565 3162 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 18:51:40.715304 3178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:40.716780 3162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:40.716929 3162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/wal
uuid: "a52cbd74ae76452d841b7d370265a5b4"
format_stamp: "Formatted at 2026-06-19 18:51:40 on dist-test-slave-fljv"
server_key: "440cc9af924651bc89f6f5da48baac92"
server_key_iv: "012e281e7a3188e658ab6b6a8e80a057"
server_key_version: "encryptionkey@0"
I20260619 18:51:40.717054 3162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:40.733108 3162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:40.736570 3162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:40.736756 3162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:40.743721 3162 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.252:41829
I20260619 18:51:40.743993 3230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.252:41829 every 8 connection(s)
I20260619 18:51:40.744719 3162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
I20260619 18:51:40.746637 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 3162
I20260619 18:51:40.746752 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781894826428600-13227-0/minicluster-data/master-2/wal/instance
I20260619 18:51:40.747020 13227 external_mini_cluster.cc:1428] Setting key 6e26e385b86c7b96a3dcdff0629086b8
I20260619 18:51:40.747819 3231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:40.752101 3231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:40.762857 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.748408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.12.234.252:41829)
0619 18:51:40.748831 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.748850 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.748887 (+ 37us) server_negotiation.cc:407] Connection header received
0619 18:51:40.749563 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.749580 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.749955 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.750301 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.751246 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.752250 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.753412 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.754047 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.754232 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.757570 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.757576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.757587 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.757618 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.759529 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.760009 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.760013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.760019 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.760155 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.760422 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.760424 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.760425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.760944 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.761110 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.761434 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.761777 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":119,"threads_started":1}
I20260619 18:51:40.763969 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.754148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.12.234.254:37199)
0619 18:51:40.754316 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.754320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.754742 (+ 422us) server_negotiation.cc:407] Connection header received
0619 18:51:40.754877 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.754880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.754928 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.754994 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.755949 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.756609 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.757741 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.758029 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.758130 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.760652 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.760656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.760659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.760686 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.762261 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.762667 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.762671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.762673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.762760 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.763018 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.763021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.763023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.763270 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.763340 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.763760 (+ 420us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.763860 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 18:51:40.764776 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.754074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:37199 (local address 127.0.0.1:43300)
0619 18:51:40.754600 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.754657 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:40.754786 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:40.755061 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:40.755067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:40.755572 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:40.755795 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.755805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.756745 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.756748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.757623 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.757633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.758147 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.758151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.758327 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.758922 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:40.758933 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:40.760516 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:40.762356 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.762365 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.762370 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.762577 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.762838 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.762840 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.762841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.762909 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.763414 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:40.763419 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:40.763536 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 18:51:40.763681 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 18:51:40.763855 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":229,"threads_started":1}
I20260619 18:51:40.765965 3231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
I20260619 18:51:40.772688 3080 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } attempt: 3
I20260619 18:51:40.774940 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.765444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.12.234.254:37199)
0619 18:51:40.765550 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.765554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.765636 (+ 82us) server_negotiation.cc:407] Connection header received
0619 18:51:40.765724 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.765726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.765770 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.765850 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.766682 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.767414 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.768464 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.768676 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.768769 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.771151 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.771155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.771161 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.771187 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.772836 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.773311 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.773314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.773316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.773401 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.773717 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.773721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.773722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.773962 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.774235 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.774556 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.774669 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:51:40.775089 3155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } attempt: 3
I20260619 18:51:40.775174 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.766446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:33109 (local address 127.0.0.1:40056)
0619 18:51:40.766687 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.766699 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:40.766781 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:40.766981 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:40.766984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:40.767145 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:40.767262 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.767269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.768248 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.768251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.768864 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.768873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.769366 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.769369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.769442 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.769804 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:40.769815 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:40.771406 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:40.773208 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.773211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.773213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.773420 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.774034 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.774037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.774039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.774123 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.774666 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:40.774669 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:40.774754 (+ 85us) client_negotiation.cc:769] Sending connection context
0619 18:51:40.774841 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 18:51:40.774960 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260619 18:51:40.775220 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.766570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.12.234.253:33109)
0619 18:51:40.766702 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.766706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.766742 (+ 36us) server_negotiation.cc:407] Connection header received
0619 18:51:40.766855 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.766857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.766897 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.766954 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.767360 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.768147 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.768970 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.769260 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.769359 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.771580 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.771583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.771585 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.771613 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.773094 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.773849 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.773852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.773854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.773932 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.774267 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.774270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.774272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.774480 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.774598 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.774912 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.775057 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 18:51:40.776105 3231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
I20260619 18:51:40.781914 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.773171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51638)
0619 18:51:40.773355 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.773369 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:40.773431 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:40.773750 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:40.773752 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:40.773916 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:40.773999 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.774004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.775021 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.775025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.775764 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.775775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.776318 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.776322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.776398 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.776869 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:40.776883 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:40.778287 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:40.780351 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.780354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.780356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.780610 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.780919 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.780922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.780924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.781008 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.781421 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:40.781424 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:40.781521 (+ 97us) client_negotiation.cc:769] Sending connection context
0619 18:51:40.781618 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 18:51:40.781747 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260619 18:51:40.782544 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.773242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51638 (local address 127.12.234.252:41829)
0619 18:51:40.773571 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.773575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.773587 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:51:40.773617 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.773619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.773660 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.773728 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.774204 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.774908 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.775891 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.776200 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.776599 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.778417 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.778421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.778424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.778446 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.780233 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.780729 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.780732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.780734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.780814 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.781116 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.781119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.781120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.781313 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.782229 (+ 916us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.782296 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.782403 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253}
I20260619 18:51:40.785308 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.776539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51666)
0619 18:51:40.776626 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.776638 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:40.776691 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:40.777758 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:40.777761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:40.777908 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:40.777993 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.778000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.778981 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.778984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.779750 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.779761 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.780332 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.780336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.780406 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.780791 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:40.780815 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:40.782100 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:40.784009 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.784012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.784014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.784181 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.784409 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.784412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.784414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.784483 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.784812 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:40.784815 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:40.784893 (+ 78us) client_negotiation.cc:769] Sending connection context
0619 18:51:40.784988 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 18:51:40.785123 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"spinlock_wait_cycles":2432}
I20260619 18:51:40.785341 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.775458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:41829 (local address 127.0.0.1:51652)
0619 18:51:40.775564 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:40.775578 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:40.775661 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:40.776133 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:40.776136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:40.776299 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:40.776391 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.776397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.777908 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.777912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.778659 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:40.778667 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.779252 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.779256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:40.779350 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.779720 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:40.779733 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:40.781137 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:40.783856 (+ 2719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.783859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.783861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.784083 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.784361 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:40.784364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:40.784366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:40.784444 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:40.784886 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:40.784889 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:40.784977 (+ 88us) client_negotiation.cc:769] Sending connection context
0619 18:51:40.785075 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 18:51:40.785199 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 18:51:40.785503 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.776869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51666 (local address 127.12.234.252:41829)
0619 18:51:40.777436 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.777447 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.777460 (+ 13us) server_negotiation.cc:407] Connection header received
0619 18:51:40.777489 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.777491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.777649 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.777727 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.778111 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.778878 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.779869 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.780222 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.782221 (+ 1999us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.782322 (+ 101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.782325 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.782328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.782343 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.783915 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.784260 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.784263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.784265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.784336 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.784552 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.784555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.784557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.784733 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.784811 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.785041 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.785187 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":509,"spinlock_wait_cycles":8064,"thread_start_us":100,"threads_started":1}
I20260619 18:51:40.786028 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.775496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.12.234.252:41829)
0619 18:51:40.775921 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.775930 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.775940 (+ 10us) server_negotiation.cc:407] Connection header received
0619 18:51:40.775964 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.775967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.776002 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.776491 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.776756 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.777512 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.778775 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.779125 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.779550 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.781267 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.781271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.781273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.781299 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.783736 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.784197 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.784200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.784202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.784275 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.784554 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.784556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.784558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.784800 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.784892 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.785826 (+ 934us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.785912 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"spinlock_wait_cycles":5888,"thread_start_us":115,"threads_started":1}
I20260619 18:51:40.786206 3080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65: Bootstrap starting.
I20260619 18:51:40.788024 3231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4: Bootstrap starting.
I20260619 18:51:40.789001 3155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9: Bootstrap starting.
I20260619 18:51:40.789088 3080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:40.789963 3080 log.cc:824] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:40.790360 3231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:40.791208 3231 log.cc:824] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:40.791471 3155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9: Neither blocks nor log segments found. Creating new log.
Jun 19 18:51:40 dist-test-slave-fljv krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
I20260619 18:51:40.792208 3155 log.cc:824] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:40.792673 3080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65: No bootstrap required, opened a new log
I20260619 18:51:40.794366 3155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9: No bootstrap required, opened a new log
I20260619 18:51:40.796116 3231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4: No bootstrap required, opened a new log
I20260619 18:51:40.796706 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.778375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.12.234.253:33109)
0619 18:51:40.778469 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.778473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.778637 (+ 164us) server_negotiation.cc:407] Connection header received
0619 18:51:40.778799 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.778801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.778841 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.778882 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.780190 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.780914 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.781972 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.782330 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.782406 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.792453 (+ 10047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.792458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.792461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.792490 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.794042 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.794688 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.794693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.794695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.794795 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.795152 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.795156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.795158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.795406 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.795605 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.796344 (+ 739us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.796482 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 18:51:40.796871 3080 raft_consensus.cc:348] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } }
I20260619 18:51:40.797145 3080 raft_consensus.cc:729] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c44edec7225c49d38a40ac5e71ce3f65, State: Initialized, Role: FOLLOWER
I20260619 18:51:40.797492 3080 consensus_queue.cc:260] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } }
I20260619 18:51:40.798061 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.798267 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:40.799059 3080 sys_catalog.cc:564] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:40.800071 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:40.800037 3231 raft_consensus.cc:348] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } }
I20260619 18:51:40.800302 3231 raft_consensus.cc:729] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52cbd74ae76452d841b7d370265a5b4, State: Initialized, Role: FOLLOWER
I20260619 18:51:40.800575 3231 consensus_queue.cc:260] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } }
I20260619 18:51:40.801000 3249 sys_catalog.cc:455] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.801136 3249 sys_catalog.cc:458] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 18:51:40.801970 3033 tablet.cc:2369] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:40.801970 3231 sys_catalog.cc:564] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:40.801429 3054 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c44edec7225c49d38a40ac5e71ce3f65"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43312
I20260619 18:51:40.802601 3054 raft_consensus.cc:482] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 18:51:40.802716 3054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:40.803435 3155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } }
I20260619 18:51:40.803695 3155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d75be56641741d3ac3a26ba237953b9, State: Initialized, Role: FOLLOWER
I20260619 18:51:40.804025 3155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } }
I20260619 18:51:40.804208 3054 raft_consensus.cc:504] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } }
I20260619 18:51:40.805415 3155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:40.805908 3054 leader_election.cc:288] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [CANDIDATE]: Term 1 election: Requested vote from peers 8d75be56641741d3ac3a26ba237953b9 (127.12.234.253:33109), a52cbd74ae76452d841b7d370265a5b4 (127.12.234.252:41829)
I20260619 18:51:40.806205 3129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c44edec7225c49d38a40ac5e71ce3f65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8d75be56641741d3ac3a26ba237953b9"
I20260619 18:51:40.806463 3129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [term 0 FOLLOWER]: Advancing to term 1
W20260619 18:51:40.806636 3253 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:51:40.806706 3253 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:40.807897 3129 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c44edec7225c49d38a40ac5e71ce3f65 in term 1.
I20260619 18:51:40.809113 3205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c44edec7225c49d38a40ac5e71ce3f65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a52cbd74ae76452d841b7d370265a5b4"
I20260619 18:51:40.809338 3205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [term 0 FOLLOWER]: Advancing to term 1
W20260619 18:51:40.809966 3266 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:51:40.810034 3266 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:40.810809 3205 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c44edec7225c49d38a40ac5e71ce3f65 in term 1.
I20260619 18:51:40.810786 3030 leader_election.cc:302] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d75be56641741d3ac3a26ba237953b9, c44edec7225c49d38a40ac5e71ce3f65; no voters:
I20260619 18:51:40.811800 3241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:40.812045 3241 raft_consensus.cc:686] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [term 1 LEADER]: Becoming Leader. State: Replica: c44edec7225c49d38a40ac5e71ce3f65, State: Running, Role: LEADER
I20260619 18:51:40.812412 3241 consensus_queue.cc:237] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } }
I20260619 18:51:40.816490 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c44edec7225c49d38a40ac5e71ce3f65. Latest consensus state: current_term: 1 leader_uuid: "c44edec7225c49d38a40ac5e71ce3f65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.816622 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:40.818351 3260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.818501 3260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:40.818902 3288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 18:51:40.820072 3012 master_runner.cc:263] No leader master found from master c44edec7225c49d38a40ac5e71ce3f65
I20260619 18:51:40.820150 3012 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 18:51:40.820346 3288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:40.823961 3087 master_runner.cc:263] No leader master found from master 8d75be56641741d3ac3a26ba237953b9
I20260619 18:51:40.824044 3087 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 18:51:40.825314 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.810712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.12.234.253:33109)
0619 18:51:40.810829 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.810833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.811754 (+ 921us) server_negotiation.cc:407] Connection header received
0619 18:51:40.811988 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.811992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.812055 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.812122 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.813246 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.813959 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.815108 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.815419 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.815493 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.820406 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.820410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.820413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.820442 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.823206 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.823842 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.823847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.823849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.823945 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.824284 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.824287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.824288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.824537 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.824639 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.825014 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.825160 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 18:51:40.828365 3282 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 18:51:40.834100 3205 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [term 1 FOLLOWER]: Refusing update from remote peer c44edec7225c49d38a40ac5e71ce3f65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:40.835096 3241 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:40.834031 3129 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [term 1 FOLLOWER]: Refusing update from remote peer c44edec7225c49d38a40ac5e71ce3f65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:40.837894 3285 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:40.839324 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.811470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.12.234.254:37199)
0619 18:51:40.811592 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.811596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.823536 (+ 11940us) server_negotiation.cc:407] Connection header received
0619 18:51:40.823674 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.823677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.823728 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.823803 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.827046 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.827837 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.829037 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.829381 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.829453 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.833482 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.833487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.833489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.833533 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.836103 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.837538 (+ 1435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.837541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.837543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.837639 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.837950 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.837953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.837954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.838223 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.838347 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.839022 (+ 675us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.839149 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 18:51:40.842506 3282 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:40.842790 3260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c44edec7225c49d38a40ac5e71ce3f65. Latest consensus state: current_term: 1 leader_uuid: "c44edec7225c49d38a40ac5e71ce3f65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.842937 3260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:40.845984 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:40.812259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51678 (local address 127.12.234.252:41829)
0619 18:51:40.812371 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.812375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.827033 (+ 14658us) server_negotiation.cc:407] Connection header received
0619 18:51:40.827190 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.827193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.827237 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.827282 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.828146 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.828894 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.831343 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:40.831741 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:40.831821 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:40.834536 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:40.834540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:40.834543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:40.834572 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:40.836988 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.840750 (+ 3762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.840754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.840757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.840880 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:40.841250 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:40.841253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:40.841255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:40.841481 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:40.845623 (+ 4142us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:40.845731 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 18:51:40.845858 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 18:51:40.849591 3249 sys_catalog.cc:455] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c44edec7225c49d38a40ac5e71ce3f65. Latest consensus state: current_term: 1 leader_uuid: "c44edec7225c49d38a40ac5e71ce3f65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.849745 3249 sys_catalog.cc:458] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:40.851275 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c44edec7225c49d38a40ac5e71ce3f65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.851416 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:40.851805 3289 mvcc.cc:205] Tried to move back new op lower bound from 7298642333006524416 to 7298642332931215360. Current Snapshot: MvccSnapshot[applied={T|T < 7298642333006524416}]
I20260619 18:51:40.853386 3260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c44edec7225c49d38a40ac5e71ce3f65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.853528 3260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:40.854029 3291 mvcc.cc:205] Tried to move back new op lower bound from 7298642333006524416 to 7298642332931215360. Current Snapshot: MvccSnapshot[applied={T|T < 7298642333006524416}]
I20260619 18:51:40.863416 3288 catalog_manager.cc:1347] Generated new cluster ID: ededdfaf997b492bab5f95be724490ce
I20260619 18:51:40.863523 3288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:40.875310 3285 sys_catalog.cc:455] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c44edec7225c49d38a40ac5e71ce3f65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.875476 3285 sys_catalog.cc:458] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:40.876904 3249 sys_catalog.cc:455] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c44edec7225c49d38a40ac5e71ce3f65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44edec7225c49d38a40ac5e71ce3f65" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 37199 } } peers { permanent_uuid: "8d75be56641741d3ac3a26ba237953b9" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 33109 } } peers { permanent_uuid: "a52cbd74ae76452d841b7d370265a5b4" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 41829 } } }
I20260619 18:51:40.877074 3249 sys_catalog.cc:458] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:40.897864 3288 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:40.899329 3288 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:40.921478 3288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c44edec7225c49d38a40ac5e71ce3f65: Generated new TSK 0
I20260619 18:51:40.922518 3288 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 18:51:40.958490 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:40.955422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40076 (local address 127.12.234.253:33109)
0619 18:51:40.955564 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.955568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.957263 (+ 1695us) server_negotiation.cc:407] Connection header received
0619 18:51:40.957471 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.957475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.957531 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.957586 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.958339 (+ 753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40076: BlockingRecv error: recv got EOF from 127.0.0.1:40076 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 18:51:40.959983 3083 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:40.956212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43336 (local address 127.12.234.254:37199)
0619 18:51:40.956342 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.956346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.959012 (+ 2666us) server_negotiation.cc:407] Connection header received
0619 18:51:40.959211 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.959215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.959269 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.959320 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.959839 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43336: BlockingRecv error: recv got EOF from 127.0.0.1:43336 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 18:51:40.961496 3236 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:40.956726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51686 (local address 127.12.234.252:41829)
0619 18:51:40.956830 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.956835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.960486 (+ 3651us) server_negotiation.cc:407] Connection header received
0619 18:51:40.960719 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.960722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.960772 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.960820 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.961370 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51686: BlockingRecv error: recv got EOF from 127.0.0.1:51686 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 18:51:40.966444 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:40.964672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40092 (local address 127.12.234.253:33109)
0619 18:51:40.964789 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.964794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.965316 (+ 522us) server_negotiation.cc:407] Connection header received
0619 18:51:40.965576 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.965579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.965664 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.965720 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.966326 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40092: BlockingRecv error: recv got EOF from 127.0.0.1:40092 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 18:51:40.971747 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:40.970095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40096 (local address 127.12.234.253:33109)
0619 18:51:40.970256 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.970260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.970796 (+ 536us) server_negotiation.cc:407] Connection header received
0619 18:51:40.970985 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.970988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.971053 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.971110 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.971633 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40096: BlockingRecv error: recv got EOF from 127.0.0.1:40096 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 18:51:40.977746 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:40.975975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40106 (local address 127.12.234.253:33109)
0619 18:51:40.976102 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.976106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.976686 (+ 580us) server_negotiation.cc:407] Connection header received
0619 18:51:40.976884 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.976887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.976942 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.976992 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.977624 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40106: BlockingRecv error: recv got EOF from 127.0.0.1:40106 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260619 18:51:40.985805 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:40.984068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40116 (local address 127.12.234.253:33109)
0619 18:51:40.984266 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.984271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.984846 (+ 575us) server_negotiation.cc:407] Connection header received
0619 18:51:40.985038 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.985041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.985096 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.985147 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.985682 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40116: BlockingRecv error: recv got EOF from 127.0.0.1:40116 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260619 18:51:40.998463 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:40.996606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.12.234.253:33109)
0619 18:51:40.996742 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:40.996746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:40.997397 (+ 651us) server_negotiation.cc:407] Connection header received
0619 18:51:40.997628 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:40.997632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:40.997692 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:40.997745 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:40.998343 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40128: BlockingRecv error: recv got EOF from 127.0.0.1:40128 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 18:51:41.018095 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:41.016869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40134 (local address 127.12.234.253:33109)
0619 18:51:41.016992 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:41.016996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:41.017248 (+ 252us) server_negotiation.cc:407] Connection header received
0619 18:51:41.017385 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:41.017388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:41.017438 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:41.017506 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:41.017971 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40134: BlockingRecv error: recv got EOF from 127.0.0.1:40134 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260619 18:51:41.053670 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:41.052358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.12.234.253:33109)
0619 18:51:41.052486 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:41.052490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:41.052784 (+ 294us) server_negotiation.cc:407] Connection header received
0619 18:51:41.052927 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:41.052931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:41.052993 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:41.053108 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:41.053545 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40144: BlockingRecv error: recv got EOF from 127.0.0.1:40144 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 18:51:41.120968 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:41.119923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40158 (local address 127.12.234.253:33109)
0619 18:51:41.120140 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:41.120144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:41.120162 (+ 18us) server_negotiation.cc:407] Connection header received
0619 18:51:41.120287 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:41.120289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:41.120339 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:41.120424 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:41.120855 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40158: BlockingRecv error: recv got EOF from 127.0.0.1:40158 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260619 18:51:41.252456 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:41.251375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40174 (local address 127.12.234.253:33109)
0619 18:51:41.251533 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:41.251538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:41.251632 (+ 94us) server_negotiation.cc:407] Connection header received
0619 18:51:41.251788 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:41.251792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:41.251838 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:41.251907 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:41.252358 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40174: BlockingRecv error: recv got EOF from 127.0.0.1:40174 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 18:51:41.511741 3161 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:41.510638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40182 (local address 127.12.234.253:33109)
0619 18:51:41.510779 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:41.510783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:41.510874 (+ 91us) server_negotiation.cc:407] Connection header received
0619 18:51:41.511044 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:41.511048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:41.511098 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:41.511172 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:41.511623 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40182: BlockingRecv error: recv got EOF from 127.0.0.1:40182 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 18:51:41.811316 3266 catalog_manager.cc:1259] Loaded cluster ID: ededdfaf997b492bab5f95be724490ce
I20260619 18:51:41.811400 3266 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4: loading cluster ID for follower catalog manager: success
I20260619 18:51:41.813161 3266 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4: acquiring CA information for follower catalog manager: success
I20260619 18:51:41.814451 3266 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a52cbd74ae76452d841b7d370265a5b4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 18:51:41.843873 3282 catalog_manager.cc:1259] Loaded cluster ID: ededdfaf997b492bab5f95be724490ce
I20260619 18:51:41.843961 3282 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9: loading cluster ID for follower catalog manager: success
I20260619 18:51:41.846098 3282 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9: acquiring CA information for follower catalog manager: success
I20260619 18:51:41.847430 3282 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8d75be56641741d3ac3a26ba237953b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 18:51:42.030359 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:42.025895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.12.234.253:33109)
0619 18:51:42.026169 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:42.026172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:42.026302 (+ 130us) server_negotiation.cc:407] Connection header received
0619 18:51:42.026443 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:42.026446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:42.026578 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:42.026677 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:42.026999 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.027564 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.028556 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.028825 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.028916 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:42.029340 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:42.029827 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:42.029926 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:42.030089 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 18:51:42.030151 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":74,"threads_started":1}
I20260619 18:51:42.038436 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:42.033973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51692 (local address 127.12.234.252:41829)
0619 18:51:42.034233 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:42.034236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:42.034277 (+ 41us) server_negotiation.cc:407] Connection header received
0619 18:51:42.034499 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:42.034502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:42.034632 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:42.034749 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:42.034993 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.035657 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.036785 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.037031 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.037127 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:42.037465 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:42.037954 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:42.038080 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:42.038144 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 18:51:42.038261 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":74,"threads_started":1}
I20260619 18:51:42.046658 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:42.041964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.12.234.254:37199)
0619 18:51:42.042298 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:42.042302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:42.042626 (+ 324us) server_negotiation.cc:407] Connection header received
0619 18:51:42.042772 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:42.042775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:42.042931 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:42.043037 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:42.043306 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.043868 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.044756 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.045005 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.045092 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:42.045515 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:42.045953 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:42.046034 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:42.046389 (+ 355us) server_negotiation.cc:299] Negotiation successful
0619 18:51:42.046485 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":64,"threads_started":1}
I20260619 18:51:42.046662 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:42.041813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40196 (local address 127.12.234.253:33109)
0619 18:51:42.041965 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:42.041970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:42.042500 (+ 530us) server_negotiation.cc:407] Connection header received
0619 18:51:42.042587 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:42.042591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:42.042660 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:42.042733 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:42.043118 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.043955 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.045515 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.045842 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.045953 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:42.046215 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:42.046299 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:42.046361 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:42.046494 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 18:51:42.046553 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 18:51:42.046702 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:42.042172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.12.234.252:41829)
0619 18:51:42.042315 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:42.042319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:42.042334 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:51:42.042530 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:42.042534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:42.042592 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:42.042632 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:42.043117 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.043953 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.045474 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:42.045746 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:42.045808 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:42.046225 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:42.046314 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:42.046364 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:42.046506 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 18:51:42.046568 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 18:51:42.054620 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 3012
I20260619 18:51:42.060067 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 3087
I20260619 18:51:42.065037 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 3162
2026-06-19T18:51:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3044 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 18:51:42 dist-test-slave-fljv krb5kdc[3379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 18:51:42 dist-test-slave-fljv krb5kdc[3379](info): set up 2 sockets
Jun 19 18:51:42 dist-test-slave-fljv krb5kdc[3379](info): commencing operation
krb5kdc: starting...
W20260619 18:51:43.026633 13227 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/test-user.keytab.
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T18:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T18:51:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
Entry for principal kudu/127.12.234.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab.
I20260619 18:51:43.149289 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:43485
--webserver_interface=127.12.234.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:43485,127.12.234.253:35375,127.12.234.252:44171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 18:51:43.264875 3392 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 18:51:43.265201 3392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:43.265226 3392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:43.265273 3392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:43.269558 3392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:43.269647 3392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:43.269666 3392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:43.269682 3392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:43.269696 3392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:43.275453 3392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:43485,127.12.234.253:35375,127.12.234.252:44171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.254.keytab
--principal=kudu/127.12.234.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.254:43485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:43.276621 3392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:43.277716 3392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 18:51:43.286823 3392 server_base.cc:1034] running on GCE node
W20260619 18:51:43.286775 3398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:43.286767 3400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:43.286775 3397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:43.287647 3392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:43.288873 3392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:43.290047 3392 hybrid_clock.cc:648] HybridClock initialized: now 1781895103290036 us; error 76 us; skew 500 ppm
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:43.292898 3392 init.cc:377] Logged in from keytab as kudu/127.12.234.254@KRBTEST.COM (short username kudu)
I20260619 18:51:43.293891 3392 webserver.cc:460] Webserver started at http://127.12.234.254:32827/ using document root <none> and password file <none>
I20260619 18:51:43.294507 3392 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:43.294580 3392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:43.294793 3392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:43.296947 3392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/data/instance:
uuid: "053529ef8b044224912e156d9f791e83"
format_stamp: "Formatted at 2026-06-19 18:51:43 on dist-test-slave-fljv"
server_key: "2d45f2c8baf61e5c125476911f9b4637"
server_key_iv: "20331f4279c291fd930fbad80ffd9870"
server_key_version: "encryptionkey@0"
I20260619 18:51:43.297513 3392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/wal/instance:
uuid: "053529ef8b044224912e156d9f791e83"
format_stamp: "Formatted at 2026-06-19 18:51:43 on dist-test-slave-fljv"
server_key: "2d45f2c8baf61e5c125476911f9b4637"
server_key_iv: "20331f4279c291fd930fbad80ffd9870"
server_key_version: "encryptionkey@0"
I20260619 18:51:43.301358 3392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:51:43.303954 3407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:43.305205 3392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 18:51:43.305373 3392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/wal
uuid: "053529ef8b044224912e156d9f791e83"
format_stamp: "Formatted at 2026-06-19 18:51:43 on dist-test-slave-fljv"
server_key: "2d45f2c8baf61e5c125476911f9b4637"
server_key_iv: "20331f4279c291fd930fbad80ffd9870"
server_key_version: "encryptionkey@0"
I20260619 18:51:43.305482 3392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:43.317298 3392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:43.320784 3392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:43.320986 3392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:43.328258 3392 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.254:43485
I20260619 18:51:43.328562 3459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.254:43485 every 8 connection(s)
I20260619 18:51:43.329298 3392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/data/info.pb
I20260619 18:51:43.332276 3460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:43.335999 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 3392
I20260619 18:51:43.336127 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-0/wal/instance
I20260619 18:51:43.336406 13227 external_mini_cluster.cc:1428] Setting key 076fd8e290dc3476387e5cbb35b16c1d
I20260619 18:51:43.336371 3460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.234.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.253@KRBTEST.COM" created.
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:43.353353 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.339234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43485 (local address 127.0.0.1:40122)
0619 18:51:43.339811 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.339856 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:43.340770 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:43.341477 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:43.341483 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:43.341976 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:43.342279 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.342295 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.343410 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.343413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.344181 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.344190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.344808 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.344812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.344941 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.345473 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:43.345490 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:43.347585 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:43.349892 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.349901 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.349911 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.350297 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.350711 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.350715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.350717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.350867 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.351704 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:43.351709 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:43.351845 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 18:51:43.351997 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 18:51:43.352546 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":195,"spinlock_wait_cycles":24448,"thread_start_us":145,"threads_started":1}
I20260619 18:51:43.353358 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.339233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.12.234.254:43485)
0619 18:51:43.339673 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.339696 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.339966 (+ 270us) server_negotiation.cc:407] Connection header received
0619 18:51:43.341086 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.341090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.341294 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.341462 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.342418 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.343293 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.344283 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.344660 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.344815 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.347751 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.347758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.347770 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.347792 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.349741 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.350405 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.350413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.350415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.350566 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.350971 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.350974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.350976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.351568 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.351721 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.352124 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.352435 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260619 18:51:43.356047 3460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:43.356806 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.356516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:35375 (local address 127.0.0.1:35538)
0619 18:51:43.356652 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.356729 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 18:51:43.357013 3409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
I20260619 18:51:43.358011 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.357733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:35375 (local address 127.0.0.1:35544)
0619 18:51:43.357869 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.357931 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 18:51:43.358245 3460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.253:35375: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab.
Entry for principal kudu/127.12.234.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab.
I20260619 18:51:43.365140 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.253:35375
--webserver_interface=127.12.234.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab
--principal=kudu/127.12.234.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:43485,127.12.234.253:35375,127.12.234.252:44171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:51:43.407552 3460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } attempt: 1
I20260619 18:51:43.408515 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.408176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:35375 (local address 127.0.0.1:35546)
0619 18:51:43.408343 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.408423 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 18:51:43.409300 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.409069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:35375 (local address 127.0.0.1:35552)
0619 18:51:43.409177 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.409232 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 18:51:43.409425 3460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.253:35375: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
I20260619 18:51:43.477607 3460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } attempt: 2
I20260619 18:51:43.478579 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.478240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:35375 (local address 127.0.0.1:35564)
0619 18:51:43.478399 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.478478 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 18:51:43.479431 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.479219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:35375 (local address 127.0.0.1:35580)
0619 18:51:43.479316 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.479369 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 18:51:43.479604 3460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.253:35375: Network error: Client connection negotiation failed: client connection to 127.12.234.253:35375: connect: Connection refused (error 111)
W20260619 18:51:43.480176 3467 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 18:51:43.480495 3467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:43.480522 3467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:43.480587 3467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 18:51:43.484993 3467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:43.485065 3467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:43.485085 3467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:43.485100 3467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:43.485114 3467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:43.490983 3467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:43485,127.12.234.253:35375,127.12.234.252:44171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.253.keytab
--principal=kudu/127.12.234.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.253:35375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.234.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:43.492213 3467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:43.493261 3467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:43.501940 3473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:43.501966 3475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:43.501966 3472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:43.502506 3467 server_base.cc:1034] running on GCE node
I20260619 18:51:43.503156 3467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:43.504432 3467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:43.505649 3467 hybrid_clock.cc:648] HybridClock initialized: now 1781895103505616 us; error 51 us; skew 500 ppm
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:43.508659 3467 init.cc:377] Logged in from keytab as kudu/127.12.234.253@KRBTEST.COM (short username kudu)
I20260619 18:51:43.509757 3467 webserver.cc:460] Webserver started at http://127.12.234.253:34751/ using document root <none> and password file <none>
I20260619 18:51:43.510421 3467 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:43.510497 3467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:43.510720 3467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:43.512918 3467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/data/instance:
uuid: "d5bf802364b14e09941c2f315f5d125d"
format_stamp: "Formatted at 2026-06-19 18:51:43 on dist-test-slave-fljv"
server_key: "16d0bf74f3cfac092ef00882cd5230a6"
server_key_iv: "05f47275adfd242dd30e40f934ebf21b"
server_key_version: "encryptionkey@0"
I20260619 18:51:43.513490 3467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/wal/instance:
uuid: "d5bf802364b14e09941c2f315f5d125d"
format_stamp: "Formatted at 2026-06-19 18:51:43 on dist-test-slave-fljv"
server_key: "16d0bf74f3cfac092ef00882cd5230a6"
server_key_iv: "05f47275adfd242dd30e40f934ebf21b"
server_key_version: "encryptionkey@0"
I20260619 18:51:43.517637 3467 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 18:51:43.520665 3482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:43.522297 3467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 18:51:43.522444 3467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/wal
uuid: "d5bf802364b14e09941c2f315f5d125d"
format_stamp: "Formatted at 2026-06-19 18:51:43 on dist-test-slave-fljv"
server_key: "16d0bf74f3cfac092ef00882cd5230a6"
server_key_iv: "05f47275adfd242dd30e40f934ebf21b"
server_key_version: "encryptionkey@0"
I20260619 18:51:43.522564 3467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:43.542219 3467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:43.545480 3467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:43.545672 3467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:43.552803 3467 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.253:35375
I20260619 18:51:43.553051 3534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.253:35375 every 8 connection(s)
I20260619 18:51:43.553853 3467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/data/info.pb
I20260619 18:51:43.557123 3535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:43.561267 3535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:43.562144 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 3467
I20260619 18:51:43.562290 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-1/wal/instance
I20260619 18:51:43.562549 13227 external_mini_cluster.cc:1428] Setting key 3cfa955ed9e5862304da22a8e7781a8c
WARNING: no policy specified for kudu/127.12.234.252@KRBTEST.COM; defaulting to no policy
I20260619 18:51:43.570801 3460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.252@KRBTEST.COM" created.
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
I20260619 18:51:43.579641 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.563326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40136 (local address 127.12.234.254:43485)
0619 18:51:43.563482 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.563486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.564542 (+ 1056us) server_negotiation.cc:407] Connection header received
0619 18:51:43.565800 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.565803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.565859 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.565936 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.567688 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.568274 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.570075 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.570398 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.570501 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.574791 (+ 4290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.574795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.574798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.574825 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.576957 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.577808 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.577811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.577813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.577892 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.578305 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.578309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.578311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.578544 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.578649 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.579345 (+ 696us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.579469 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56,"spinlock_wait_cycles":32512}
I20260619 18:51:43.580631 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.563739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43485 (local address 127.0.0.1:40136)
0619 18:51:43.564324 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.564396 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:43.565535 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:43.566150 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:43.566160 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:43.566768 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:43.567496 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.567519 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.568424 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.568428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.569863 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.569876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.570576 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.570581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.570784 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.571462 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:43.571485 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:43.574635 (+ 3150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:43.577085 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.577092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.577110 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.577458 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.577984 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.577987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.577989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.578166 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.578665 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:43.578672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:43.579045 (+ 373us) client_negotiation.cc:769] Sending connection context
0619 18:51:43.579257 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 18:51:43.579589 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":131,"threads_started":1}
I20260619 18:51:43.580818 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.571271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:35375 (local address 127.0.0.1:35596)
0619 18:51:43.571463 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.571475 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:43.571553 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:43.572176 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:43.572179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:43.572320 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:43.572422 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.572427 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.573407 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.573410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.574004 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.574014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.574681 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.574685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.574795 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.575224 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:43.575240 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:43.577191 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:43.579158 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.579161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.579164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.579401 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.579755 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.579758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.579761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.579849 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.580318 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:43.580321 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:43.580425 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 18:51:43.580528 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 18:51:43.580660 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260619 18:51:43.580900 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.571368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.12.234.253:35375)
0619 18:51:43.571736 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.571746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.571766 (+ 20us) server_negotiation.cc:407] Connection header received
0619 18:51:43.571800 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.571805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.572009 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.572176 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.572529 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.573304 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.574142 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.574567 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.574662 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.577320 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.577329 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.577341 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.577363 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.579050 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.579532 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.579539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.579541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.579630 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.579928 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.579931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.579933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.580201 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.580336 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.580617 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.580757 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":125,"threads_started":1}
I20260619 18:51:43.582036 3535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:43.583022 3460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:43.583678 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.583437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51314)
0619 18:51:43.583551 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.583605 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 18:51:43.584452 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.584187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51322)
0619 18:51:43.584362 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.584404 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260619 18:51:43.584566 3460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:44171: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
I20260619 18:51:43.590498 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.582636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35598 (local address 127.12.234.253:35375)
0619 18:51:43.582722 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.582726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.582787 (+ 61us) server_negotiation.cc:407] Connection header received
0619 18:51:43.582904 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.582906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.582937 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.582993 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.583450 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.584128 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.585187 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.585453 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.585531 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.587460 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.587463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.587465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.587486 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.588846 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.589264 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.589268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.589270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.589345 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.589642 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.589645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.589647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.589840 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.589933 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.590245 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.590345 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 18:51:43.590498 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.582570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:35375 (local address 127.0.0.1:35598)
0619 18:51:43.582720 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.582745 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:43.582826 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:43.583077 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:43.583079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:43.583233 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:43.583345 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.583353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.584250 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.584254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.585053 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.585090 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.585545 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.585563 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.585662 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.586040 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:43.586051 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:43.587334 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:43.588965 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.588968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.588970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.589167 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.589430 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.589433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.589435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.589533 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.589935 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:43.589938 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:43.590026 (+ 88us) client_negotiation.cc:769] Sending connection context
0619 18:51:43.590125 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 18:51:43.590294 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260619 18:51:43.591253 3535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab.
Entry for principal kudu/127.12.234.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab.
I20260619 18:51:43.591940 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.591664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51332)
0619 18:51:43.591778 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.591872 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 18:51:43.592518 3485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
I20260619 18:51:43.592782 13227 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
/tmp/dist-test-taskJa0qyg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.252:44171
--webserver_interface=127.12.234.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab
--principal=kudu/127.12.234.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.212:43223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.254:43485,127.12.234.253:35375,127.12.234.252:44171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 18:51:43.593371 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.593167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51340)
0619 18:51:43.593271 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.593317 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 18:51:43.593995 3535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:44171: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
I20260619 18:51:43.615733 3460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } attempt: 1
I20260619 18:51:43.616677 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.616351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51348)
0619 18:51:43.616523 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.616594 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 18:51:43.617430 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.617212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51352)
0619 18:51:43.617318 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.617367 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 18:51:43.617555 3460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:44171: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
I20260619 18:51:43.643260 3535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } attempt: 1
I20260619 18:51:43.644316 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.643888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51368)
0619 18:51:43.644138 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.644214 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260619 18:51:43.645215 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.644942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51380)
0619 18:51:43.645093 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.645144 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260619 18:51:43.645387 3535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:44171: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
I20260619 18:51:43.692687 3460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } attempt: 2
I20260619 18:51:43.693562 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.693268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51386)
0619 18:51:43.693416 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.693478 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 18:51:43.694355 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.694099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51394)
0619 18:51:43.694245 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.694290 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 18:51:43.694497 3460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:44171: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
W20260619 18:51:43.712182 3542 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 18:51:43.712527 3542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 18:51:43.712569 3542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 18:51:43.712643 3542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 18:51:43.713573 3535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } attempt: 2
I20260619 18:51:43.714509 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.714147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51396)
0619 18:51:43.714349 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.714425 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260619 18:51:43.715322 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.715046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51406)
0619 18:51:43.715197 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.715251 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260619 18:51:43.715456 3535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.252:44171: Network error: Client connection negotiation failed: client connection to 127.12.234.252:44171: connect: Connection refused (error 111)
W20260619 18:51:43.717289 3542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 18:51:43.717361 3542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 18:51:43.717384 3542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 18:51:43.717403 3542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 18:51:43.717429 3542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 18:51:43.723441 3542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.212:43223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.254:43485,127.12.234.253:35375,127.12.234.252:44171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/krb5kdc/kudu_127.12.234.252.keytab
--principal=kudu/127.12.234.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.252:44171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.234.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 17:59:53 UTC on ccycloud.quasar-efnmxv.root.comops.site
build id 4456541
I20260619 18:51:43.724575 3542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 18:51:43.725672 3542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 18:51:43.735435 3548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:43.735435 3542 server_base.cc:1034] running on GCE node
W20260619 18:51:43.735476 3550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 18:51:43.735445 3547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 18:51:43.736304 3542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 18:51:43.737515 3542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 18:51:43.738684 3542 hybrid_clock.cc:648] HybridClock initialized: now 1781895103738667 us; error 54 us; skew 500 ppm
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 18:51:43.741456 3542 init.cc:377] Logged in from keytab as kudu/127.12.234.252@KRBTEST.COM (short username kudu)
I20260619 18:51:43.742511 3542 webserver.cc:460] Webserver started at http://127.12.234.252:35587/ using document root <none> and password file <none>
I20260619 18:51:43.743078 3542 fs_manager.cc:359] Metadata directory not provided
I20260619 18:51:43.743150 3542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 18:51:43.743346 3542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 18:51:43.745476 3542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/data/instance:
uuid: "96fdd23b3bb14969a21c8df81ffbc0e0"
format_stamp: "Formatted at 2026-06-19 18:51:43 on dist-test-slave-fljv"
server_key: "2078fa43814d5daa0e467b956bb4925a"
server_key_iv: "b7a9f423f8a63faf370f5f19216a47e4"
server_key_version: "encryptionkey@0"
I20260619 18:51:43.746029 3542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/wal/instance:
uuid: "96fdd23b3bb14969a21c8df81ffbc0e0"
format_stamp: "Formatted at 2026-06-19 18:51:43 on dist-test-slave-fljv"
server_key: "2078fa43814d5daa0e467b956bb4925a"
server_key_iv: "b7a9f423f8a63faf370f5f19216a47e4"
server_key_version: "encryptionkey@0"
I20260619 18:51:43.749926 3542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 18:51:43.752708 3557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 18:51:43.754129 3542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 18:51:43.754323 3542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/data,/tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/wal
uuid: "96fdd23b3bb14969a21c8df81ffbc0e0"
format_stamp: "Formatted at 2026-06-19 18:51:43 on dist-test-slave-fljv"
server_key: "2078fa43814d5daa0e467b956bb4925a"
server_key_iv: "b7a9f423f8a63faf370f5f19216a47e4"
server_key_version: "encryptionkey@0"
I20260619 18:51:43.754443 3542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 18:51:43.769754 3542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 18:51:43.773109 3542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 18:51:43.773307 3542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 18:51:43.780143 3542 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.252:44171
I20260619 18:51:43.780454 3609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.252:44171 every 8 connection(s)
I20260619 18:51:43.781139 3542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/data/info.pb
I20260619 18:51:43.784108 3610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 18:51:43.788117 3610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:43.789535 13227 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu as pid 3542
I20260619 18:51:43.789659 13227 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJa0qyg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781894826428600-13227-0/minicluster-data/master-2/wal/instance
I20260619 18:51:43.789968 13227 external_mini_cluster.cc:1428] Setting key 0a52d069ab677780246c51bf419eb870
I20260619 18:51:43.793658 3460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } attempt: 3
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.254@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
I20260619 18:51:43.802647 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.790293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40138 (local address 127.12.234.254:43485)
0619 18:51:43.790442 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.790447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.791418 (+ 971us) server_negotiation.cc:407] Connection header received
0619 18:51:43.792424 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.792428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.792484 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.792552 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.793400 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.794002 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.795329 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.795535 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.795643 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.798785 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.798790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.798792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.798819 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.800407 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.800900 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.800904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.800906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.800980 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.801394 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.801398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.801399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.801610 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.801703 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.802348 (+ 645us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.802469 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 18:51:43.803072 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.794037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51432)
0619 18:51:43.794472 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.794486 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:43.794572 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:43.794815 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:43.794818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:43.794998 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:43.795089 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.795094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.796002 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.796005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.796688 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.796700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.797200 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.797203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.797288 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.797683 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:43.797696 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:43.799011 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:43.801432 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.801435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.801436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.801617 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.801920 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.801923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.801924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.802013 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.802555 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:43.802558 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:43.802654 (+ 96us) client_negotiation.cc:769] Sending connection context
0619 18:51:43.802737 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 18:51:43.802865 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355}
I20260619 18:51:43.803687 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.790738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.254:43485 (local address 127.0.0.1:40138)
0619 18:51:43.791261 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.791317 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:43.792209 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:43.792625 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:43.792631 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:43.793000 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:43.793258 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.793271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.794107 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.794110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.795234 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.795243 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.795672 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.795676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.795863 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.796728 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:43.796749 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:43.798624 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:43.800507 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.800517 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.800530 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.800798 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.801058 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.801061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.801063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.801253 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.801722 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:43.801728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:43.802050 (+ 322us) client_negotiation.cc:769] Sending connection context
0619 18:51:43.802298 (+ 248us) client_negotiation.cc:240] Negotiation successful
0619 18:51:43.802636 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":162,"threads_started":1}
I20260619 18:51:43.803823 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.794221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51432 (local address 127.12.234.252:44171)
0619 18:51:43.794494 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.794502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.794532 (+ 30us) server_negotiation.cc:407] Connection header received
0619 18:51:43.794655 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.794657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.794697 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.794774 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.795196 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.795902 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.796820 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.797071 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.797176 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.799394 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.799400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.799412 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.799433 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.801331 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.801728 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.801734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.801735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.801828 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.802103 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.802107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.802109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.802445 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.803394 (+ 949us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.803485 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.803620 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
I20260619 18:51:43.804919 3610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:43.805711 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.790752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.12.234.252:44171)
0619 18:51:43.791730 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.791738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.791770 (+ 32us) server_negotiation.cc:407] Connection header received
0619 18:51:43.792218 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.792236 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.792453 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.792662 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.793446 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.794617 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.795682 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.796184 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.796278 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.799796 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.799800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.799802 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.799823 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.802333 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.802845 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.802848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.802850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.803035 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.803337 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.803340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.803342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.803529 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.803711 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.805267 (+ 1556us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.805375 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"spinlock_wait_cycles":60672,"thread_start_us":181,"threads_started":1}
I20260619 18:51:43.806594 3535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } attempt: 3
I20260619 18:51:43.808337 3460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83: Bootstrap starting.
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
I20260619 18:51:43.811342 3460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:43.812306 3460 log.cc:824] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83: Log is configured to *not* fsync() on all Append() calls
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.253@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.254@KRBTEST.COM
I20260619 18:51:43.814674 3460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83: No bootstrap required, opened a new log
I20260619 18:51:43.815791 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.805474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.12.234.253:35375)
0619 18:51:43.805602 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.805607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.805621 (+ 14us) server_negotiation.cc:407] Connection header received
0619 18:51:43.805664 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.805667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.805712 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.805781 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.806384 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.807139 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.808111 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.808459 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.808532 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.811564 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.811569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.811571 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.811599 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.813271 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.813707 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.813710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.813712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.813795 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.814859 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.814863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.814865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.815088 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.815158 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.815499 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.815616 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 18:51:43.816110 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.805365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.253:35375 (local address 127.0.0.1:35602)
0619 18:51:43.805505 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.805517 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:43.805586 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:43.805978 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:43.805981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:43.806140 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:43.806273 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.806280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.807240 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.807244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.807971 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.807981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.808773 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.808776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.808884 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.809403 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:43.809418 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:43.810972 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:43.813366 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.813369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.813371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.813597 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.814295 (+ 698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.814299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.814301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.814398 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.815227 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:43.815229 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:43.815323 (+ 94us) client_negotiation.cc:769] Sending connection context
0619 18:51:43.815826 (+ 503us) client_negotiation.cc:240] Negotiation successful
0619 18:51:43.815969 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260619 18:51:43.816955 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.806932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51438)
0619 18:51:43.807024 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.807038 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:43.807109 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:43.807999 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:43.808002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:43.808186 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:43.808282 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.808288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.809368 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.809371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.810231 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.810243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.810831 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.810834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.810912 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.811370 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:43.811384 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:43.813555 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:43.815295 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.815298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.815300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.815514 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.815785 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.815788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.815790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.815882 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.816503 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:43.816506 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:43.816598 (+ 92us) client_negotiation.cc:769] Sending connection context
0619 18:51:43.816681 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 18:51:43.816810 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 18:51:43.817118 3610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } has no permanent_uuid. Determining permanent_uuid...
I20260619 18:51:43.817804 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.807143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51438 (local address 127.12.234.252:44171)
0619 18:51:43.807714 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.807718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.807730 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:51:43.807798 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.807801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.807840 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.808392 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.808523 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.809243 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.810399 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.810705 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.811119 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.813695 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.813699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.813702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.813723 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.815198 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.815619 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.815622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.815623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.815695 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.816171 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.816175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.816176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.816400 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.817471 (+ 1071us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.817548 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.817658 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480}
I20260619 18:51:43.819614 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.807311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.12.234.254:43485)
0619 18:51:43.807435 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.807439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.807511 (+ 72us) server_negotiation.cc:407] Connection header received
0619 18:51:43.807628 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.807631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.807674 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.807742 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.808548 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.809301 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.810440 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.810750 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.810822 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.813688 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.813691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.813694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.813719 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.815357 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.815836 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.815839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.815841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.815921 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.816597 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.816601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.816602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.816819 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.817007 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.819315 (+ 2308us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.819431 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 18:51:43.819449 3460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } }
I20260619 18:51:43.819712 3460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053529ef8b044224912e156d9f791e83, State: Initialized, Role: FOLLOWER
I20260619 18:51:43.820065 3460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } }
I20260619 18:51:43.820586 3617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.820691 3535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d: Bootstrap starting.
I20260619 18:51:43.820732 3617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:43.821540 3460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 18:51:43.823179 3413 tablet.cc:2369] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 18:51:43.823359 3535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:43.824247 3535 log.cc:824] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d: Log is configured to *not* fsync() on all Append() calls
W20260619 18:51:43.826778 3628 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.253@KRBTEST.COM
W20260619 18:51:43.828229 3628 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 19 18:51:43 dist-test-slave-fljv krb5kdc[3379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781895103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.252@KRBTEST.COM for kudu/127.12.234.252@KRBTEST.COM
I20260619 18:51:43.826346 3392 master_runner.cc:186] Error getting master registration for 127.12.234.253:35375: OK, instance_id { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" instance_seqno: 1781895103550746 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 18:51:43.830482 3392 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 18:51:43.832406 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.817625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.252:44171 (local address 127.0.0.1:51454)
0619 18:51:43.817953 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 18:51:43.817966 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 18:51:43.818033 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 18:51:43.818364 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 18:51:43.818367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 18:51:43.824821 (+ 6454us) client_negotiation.cc:189] Negotiated authn=SASL
0619 18:51:43.824938 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.824946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.825897 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.825901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.826729 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 18:51:43.826740 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.827290 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.827293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 18:51:43.827380 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.827859 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 18:51:43.827873 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 18:51:43.829212 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 18:51:43.830944 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.830947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.830949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.831167 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.831418 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 18:51:43.831421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 18:51:43.831423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 18:51:43.831488 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 18:51:43.831941 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 18:51:43.831943 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 18:51:43.832020 (+ 77us) client_negotiation.cc:769] Sending connection context
0619 18:51:43.832110 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 18:51:43.832245 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260619 18:51:43.832697 3535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d: No bootstrap required, opened a new log
I20260619 18:51:43.832970 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.817858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51454 (local address 127.12.234.252:44171)
0619 18:51:43.818081 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.818085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.818095 (+ 10us) server_negotiation.cc:407] Connection header received
0619 18:51:43.818120 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.818122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.818165 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.818279 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.825069 (+ 6790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.825786 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.826929 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.827163 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.827249 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.829341 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.829345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.829347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.829370 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.830851 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.831272 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.831274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.831276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.831345 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.831572 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.831574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.831576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.831822 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.831910 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.832736 (+ 826us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.832845 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260619 18:51:43.833078 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.820956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.12.234.253:35375)
0619 18:51:43.821066 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.821070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.821151 (+ 81us) server_negotiation.cc:407] Connection header received
0619 18:51:43.821306 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.821309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.821354 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.821431 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.822238 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.822975 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.824664 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.824985 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.825069 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.828252 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.828257 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.828260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.828287 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.830677 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.831216 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.831220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.831221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.831335 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.831662 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.831665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.831667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.831888 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.832737 (+ 849us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.832813 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.832927 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:51:43.836001 3610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0: Bootstrap starting.
I20260619 18:51:43.836650 13227 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 18:51:43.837956 3434 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "053529ef8b044224912e156d9f791e83"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40144
I20260619 18:51:43.838265 3434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 18:51:43.838406 3434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:43.838824 3610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0: Neither blocks nor log segments found. Creating new log.
I20260619 18:51:43.839766 3610 log.cc:824] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0: Log is configured to *not* fsync() on all Append() calls
I20260619 18:51:43.839826 3434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } }
I20260619 18:51:43.840915 3434 leader_election.cc:288] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [CANDIDATE]: Term 1 election: Requested vote from peers d5bf802364b14e09941c2f315f5d125d (127.12.234.253:35375), 96fdd23b3bb14969a21c8df81ffbc0e0 (127.12.234.252:44171)
I20260619 18:51:43.844166 3535 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } }
I20260619 18:51:43.844442 3535 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bf802364b14e09941c2f315f5d125d, State: Initialized, Role: FOLLOWER
I20260619 18:51:43.844800 3535 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } }
I20260619 18:51:43.847817 3610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0: No bootstrap required, opened a new log
I20260619 18:51:43.852136 3610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } }
I20260619 18:51:43.852440 3610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96fdd23b3bb14969a21c8df81ffbc0e0, State: Initialized, Role: FOLLOWER
I20260619 18:51:43.852804 3610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } }
I20260619 18:51:43.853407 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.853611 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:43.854122 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.843886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40152 (local address 127.12.234.254:43485)
0619 18:51:43.844006 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.844011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.844023 (+ 12us) server_negotiation.cc:407] Connection header received
0619 18:51:43.844057 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.844059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.844096 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.844164 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.845121 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.845838 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.847069 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.847366 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.847469 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.849252 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.849256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.849259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.849287 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.851079 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.851877 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.851880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.851882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.851970 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.852534 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.852538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.852540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.852785 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.852889 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.853559 (+ 670us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.853878 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 18:51:43.854333 3535 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:43.854401 3509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "053529ef8b044224912e156d9f791e83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d5bf802364b14e09941c2f315f5d125d"
I20260619 18:51:43.854676 3509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:43.856089 3509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 053529ef8b044224912e156d9f791e83 in term 1.
I20260619 18:51:43.856462 3610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 18:51:43.856724 3409 leader_election.cc:302] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 053529ef8b044224912e156d9f791e83, d5bf802364b14e09941c2f315f5d125d; no voters:
I20260619 18:51:43.857270 3617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 18:51:43.857518 3617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [term 1 LEADER]: Becoming Leader. State: Replica: 053529ef8b044224912e156d9f791e83, State: Running, Role: LEADER
I20260619 18:51:43.857928 3617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } }
I20260619 18:51:43.858431 3637 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.858565 3637 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:43.859978 3617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053529ef8b044224912e156d9f791e83. Latest consensus state: current_term: 1 leader_uuid: "053529ef8b044224912e156d9f791e83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.860083 3617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:43.860819 3584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "053529ef8b044224912e156d9f791e83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0"
W20260619 18:51:43.865628 3657 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:51:43.865720 3657 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:43.863060 3584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 18:51:43.866781 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.850570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.12.234.253:35375)
0619 18:51:43.850706 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.850710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.856812 (+ 6102us) server_negotiation.cc:407] Connection header received
0619 18:51:43.856998 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.857001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.857054 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.857107 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.857982 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.858806 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.860061 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.860420 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.860492 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.862313 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.862318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.862320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.862348 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.864808 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.865307 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.865311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.865313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.865405 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.865711 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.865714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.865716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.865940 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.866060 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.866479 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.866608 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 18:51:43.867852 3584 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 053529ef8b044224912e156d9f791e83 in term 1.
I20260619 18:51:43.871170 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:43.851034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51460 (local address 127.12.234.252:44171)
0619 18:51:43.851148 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:43.851153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:43.857285 (+ 6132us) server_negotiation.cc:407] Connection header received
0619 18:51:43.862384 (+ 5099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:43.862388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:43.862442 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:43.862493 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:43.863282 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.864012 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.865094 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:43.865406 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:43.865476 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:43.867492 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 18:51:43.867496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 18:51:43.867498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 18:51:43.867525 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 18:51:43.869236 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.869752 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.869756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.869757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.869844 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 18:51:43.870161 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 18:51:43.870165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 18:51:43.870167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 18:51:43.870427 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 18:51:43.870532 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:43.870890 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 18:51:43.871011 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 18:51:43.874993 3659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260619 18:51:43.875948 3665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 18:51:43.876111 3665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 18:51:43.884691 3659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 18:51:43.894752 3584 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [term 1 FOLLOWER]: Refusing update from remote peer 053529ef8b044224912e156d9f791e83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:43.895648 3655 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:43.900867 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053529ef8b044224912e156d9f791e83. Latest consensus state: current_term: 1 leader_uuid: "053529ef8b044224912e156d9f791e83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.901021 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:43.905092 3509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [term 1 FOLLOWER]: Refusing update from remote peer 053529ef8b044224912e156d9f791e83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 18:51:43.905787 3655 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 18:51:43.913782 3637 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053529ef8b044224912e156d9f791e83. Latest consensus state: current_term: 1 leader_uuid: "053529ef8b044224912e156d9f791e83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.913930 3637 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:43.920245 3655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "053529ef8b044224912e156d9f791e83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.920436 3655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:43.925043 3637 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "053529ef8b044224912e156d9f791e83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.925191 3637 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:43.925724 3668 mvcc.cc:205] Tried to move back new op lower bound from 7298642345531105280 to 7298642345406291968. Current Snapshot: MvccSnapshot[applied={T|T < 7298642345531105280}]
I20260619 18:51:43.926352 3671 mvcc.cc:205] Tried to move back new op lower bound from 7298642345531105280 to 7298642345406291968. Current Snapshot: MvccSnapshot[applied={T|T < 7298642345531105280}]
I20260619 18:51:43.931103 3617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "053529ef8b044224912e156d9f791e83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.931260 3617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83 [sys.catalog]: This master's current role is: LEADER
I20260619 18:51:43.933269 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "053529ef8b044224912e156d9f791e83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053529ef8b044224912e156d9f791e83" member_type: VOTER last_known_addr { host: "127.12.234.254" port: 43485 } } peers { permanent_uuid: "d5bf802364b14e09941c2f315f5d125d" member_type: VOTER last_known_addr { host: "127.12.234.253" port: 35375 } } peers { permanent_uuid: "96fdd23b3bb14969a21c8df81ffbc0e0" member_type: VOTER last_known_addr { host: "127.12.234.252" port: 44171 } } }
I20260619 18:51:43.933408 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 18:51:43.938663 3669 mvcc.cc:205] Tried to move back new op lower bound from 7298642345531105280 to 7298642345406291968. Current Snapshot: MvccSnapshot[applied={T|T < 7298642345531105280}]
I20260619 18:51:43.941064 3659 catalog_manager.cc:1347] Generated new cluster ID: 77aeacc9eaeb46119869a5d06ea7b09f
I20260619 18:51:43.941156 3659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 18:51:43.966660 3659 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 18:51:43.968031 3659 catalog_manager.cc:1504] Loading token signing keys...
I20260619 18:51:43.995002 3659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 053529ef8b044224912e156d9f791e83: Generated new TSK 0
I20260619 18:51:43.995966 3659 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 18:51:44.035735 3614 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.032041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.12.234.252:44171)
0619 18:51:44.032187 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.032192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.034482 (+ 2290us) server_negotiation.cc:407] Connection header received
0619 18:51:44.034738 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.034742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.034800 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.034861 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.035581 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51476: BlockingRecv error: recv got EOF from 127.0.0.1:51476 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260619 18:51:44.037433 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.032993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.12.234.253:35375)
0619 18:51:44.033119 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.033124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.036448 (+ 3324us) server_negotiation.cc:407] Connection header received
0619 18:51:44.036645 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.036648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.036705 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.036757 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.037298 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35616: BlockingRecv error: recv got EOF from 127.0.0.1:35616 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 18:51:44.039242 3463 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.033782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40158 (local address 127.12.234.254:43485)
0619 18:51:44.033899 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.033903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.037981 (+ 4078us) server_negotiation.cc:407] Connection header received
0619 18:51:44.038398 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.038402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.038454 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.038503 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.039106 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40158: BlockingRecv error: recv got EOF from 127.0.0.1:40158 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 18:51:44.044250 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.042555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35622 (local address 127.12.234.253:35375)
0619 18:51:44.042676 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.042680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.043208 (+ 528us) server_negotiation.cc:407] Connection header received
0619 18:51:44.043432 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.043435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.043521 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.043589 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.044128 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35622: BlockingRecv error: recv got EOF from 127.0.0.1:35622 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 18:51:44.050081 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.048342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.12.234.253:35375)
0619 18:51:44.048454 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.048458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.049060 (+ 602us) server_negotiation.cc:407] Connection header received
0619 18:51:44.049279 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.049283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.049343 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.049398 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.049961 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35636: BlockingRecv error: recv got EOF from 127.0.0.1:35636 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 18:51:44.056674 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.054987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.12.234.253:35375)
0619 18:51:44.055128 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.055133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.055755 (+ 622us) server_negotiation.cc:407] Connection header received
0619 18:51:44.055882 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.055885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.055947 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.056039 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.056544 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35642: BlockingRecv error: recv got EOF from 127.0.0.1:35642 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 18:51:44.064368 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.063371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.12.234.253:35375)
0619 18:51:44.063527 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.063530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.063557 (+ 27us) server_negotiation.cc:407] Connection header received
0619 18:51:44.063614 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.063617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.063670 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.063751 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.064244 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35646: BlockingRecv error: recv got EOF from 127.0.0.1:35646 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260619 18:51:44.076400 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.075178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.12.234.253:35375)
0619 18:51:44.075320 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.075324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.075537 (+ 213us) server_negotiation.cc:407] Connection header received
0619 18:51:44.075698 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.075701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.075766 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.075858 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.076282 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35658: BlockingRecv error: recv got EOF from 127.0.0.1:35658 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 18:51:44.095809 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.094612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.12.234.253:35375)
0619 18:51:44.094804 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.094809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.094913 (+ 104us) server_negotiation.cc:407] Connection header received
0619 18:51:44.095074 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.095078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.095137 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.095213 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.095706 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35666: BlockingRecv error: recv got EOF from 127.0.0.1:35666 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260619 18:51:44.131331 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.130070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35676 (local address 127.12.234.253:35375)
0619 18:51:44.130313 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.130318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.130463 (+ 145us) server_negotiation.cc:407] Connection header received
0619 18:51:44.130619 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.130623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.130675 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.130748 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.131233 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35676: BlockingRecv error: recv got EOF from 127.0.0.1:35676 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
W20260619 18:51:44.198899 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.197693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.12.234.253:35375)
0619 18:51:44.197851 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.197856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.197951 (+ 95us) server_negotiation.cc:407] Connection header received
0619 18:51:44.198123 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.198126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.198218 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.198307 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.198790 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35682: BlockingRecv error: recv got EOF from 127.0.0.1:35682 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260619 18:51:44.330329 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.329131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.12.234.253:35375)
0619 18:51:44.329255 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.329259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.329434 (+ 175us) server_negotiation.cc:407] Connection header received
0619 18:51:44.329583 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.329586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.329637 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.329708 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.330211 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35688: BlockingRecv error: recv got EOF from 127.0.0.1:35688 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 18:51:44.589890 3539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 18:51:44.588846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.12.234.253:35375)
0619 18:51:44.588967 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:44.588971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:44.588987 (+ 16us) server_negotiation.cc:407] Connection header received
0619 18:51:44.589113 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:44.589116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:44.589165 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:44.589235 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 18:51:44.589771 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35700: BlockingRecv error: recv got EOF from 127.0.0.1:35700 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 18:51:44.867085 3657 catalog_manager.cc:1259] Loaded cluster ID: 77aeacc9eaeb46119869a5d06ea7b09f
I20260619 18:51:44.867175 3657 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d: loading cluster ID for follower catalog manager: success
I20260619 18:51:44.868848 3657 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d: acquiring CA information for follower catalog manager: success
I20260619 18:51:44.870121 3657 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d5bf802364b14e09941c2f315f5d125d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 18:51:44.879719 3665 catalog_manager.cc:1259] Loaded cluster ID: 77aeacc9eaeb46119869a5d06ea7b09f
I20260619 18:51:44.879804 3665 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0: loading cluster ID for follower catalog manager: success
I20260619 18:51:44.881492 3665 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0: acquiring CA information for follower catalog manager: success
I20260619 18:51:44.882702 3665 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 96fdd23b3bb14969a21c8df81ffbc0e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 18:51:45.108417 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:45.104082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.12.234.253:35375)
0619 18:51:45.104350 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:45.104353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:45.104392 (+ 39us) server_negotiation.cc:407] Connection header received
0619 18:51:45.104541 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:45.104543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:45.104683 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:45.104804 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:45.105254 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.105836 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.106853 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.107094 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.107199 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:45.107532 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:45.107998 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:45.108123 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:45.108180 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 18:51:45.108249 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":110,"threads_started":1}
I20260619 18:51:45.116245 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:45.111837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.12.234.252:44171)
0619 18:51:45.112073 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:45.112077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:45.112092 (+ 15us) server_negotiation.cc:407] Connection header received
0619 18:51:45.112469 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:45.112472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:45.112614 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:45.112734 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:45.113046 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.113647 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.114692 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.114917 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.115005 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:45.115380 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:45.115831 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:45.115942 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:45.116008 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 18:51:45.116071 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":64,"threads_started":1}
I20260619 18:51:45.123891 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:45.119515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35720 (local address 127.12.234.253:35375)
0619 18:51:45.119633 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:45.119637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:45.120003 (+ 366us) server_negotiation.cc:407] Connection header received
0619 18:51:45.120177 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:45.120180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:45.120230 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:45.120287 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:45.120649 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.121240 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.122654 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.122965 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.123068 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:45.123408 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:45.123489 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:45.123547 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:45.123682 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 18:51:45.123748 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 18:51:45.124286 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:45.119501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.12.234.252:44171)
0619 18:51:45.119622 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:45.119626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:45.120080 (+ 454us) server_negotiation.cc:407] Connection header received
0619 18:51:45.120173 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:45.120177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:45.120230 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:45.120302 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:45.120544 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.121336 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.123217 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.123514 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.123602 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:45.123837 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:45.123911 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:45.123969 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:45.124090 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 18:51:45.124148 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 18:51:45.124545 3753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 18:51:45.119504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40168 (local address 127.12.234.254:43485)
0619 18:51:45.119743 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 18:51:45.119748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 18:51:45.119945 (+ 197us) server_negotiation.cc:407] Connection header received
0619 18:51:45.120229 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 18:51:45.120232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 18:51:45.120396 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 18:51:45.120500 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 18:51:45.120860 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.121595 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.122719 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 18:51:45.122968 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 18:51:45.123064 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 18:51:45.123706 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 18:51:45.124087 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 18:51:45.124179 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 18:51:45.124330 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 18:51:45.124407 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
I20260619 18:51:45.127827 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 3392
I20260619 18:51:45.133677 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 3467
I20260619 18:51:45.138988 13227 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJa0qyg/build/debug/bin/kudu with pid 3542
2026-06-19T18:51:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3073 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6118 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (278703 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST